2023-12-07T16:48:09.269 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-07T16:48:09.269 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193 --verbose --description rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} -- /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/orig.config.yaml 2023-12-07T16:48:09.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-07T16:48:09.377 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193 branch: wip-yuri8-testing-2023-12-06-1425 description: rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: null first_in_suite: false job_id: '7482193' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri8-testing-2023-12-06-1425 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 kernel: hwe: true workunit: branch: wip-yuri8-testing-2023-12-06-1425 sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4465 sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri8-testing-2023-12-06-1425 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-07_16:37:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-07T16:48:09.377 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa; will attempt to use it 2023-12-07T16:48:09.378 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks 2023-12-07T16:48:09.378 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-07T16:48:09.380 INFO:teuthology.task.internal:Checking packages... 2023-12-07T16:48:09.406 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e068ebcf7ecc6503f24666fb6b152034d3fe1067' 2023-12-07T16:48:09.406 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-07T16:48:09.406 INFO:teuthology.packaging:ref: None 2023-12-07T16:48:09.406 INFO:teuthology.packaging:tag: None 2023-12-07T16:48:09.406 INFO:teuthology.packaging:branch: wip-yuri8-testing-2023-12-06-1425 2023-12-07T16:48:09.406 INFO:teuthology.packaging:sha1: e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T16:48:09.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri8-testing-2023-12-06-1425 2023-12-07T16:48:09.642 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-64-ge068ebcf-1focal 2023-12-07T16:48:09.642 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-07T16:48:09.651 INFO:teuthology.task.internal:no buildpackages task found 2023-12-07T16:48:09.651 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-07T16:48:09.823 INFO:teuthology.task.internal:Saving configuration 2023-12-07T16:48:09.834 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-07T16:48:09.884 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T16:48:09.910 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 16:39:31.238400', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-12-07T16:48:09.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 16:39:31.237159', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-12-07T16:48:09.935 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-07T16:48:09.959 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2023-12-07T16:48:09.959 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2023-12-07T16:48:09.959 INFO:teuthology.run_tasks:Running task console_log... 2023-12-07T16:48:10.105 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb128d9e040>, signals=[15]) 2023-12-07T16:48:10.105 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-07T16:48:10.115 INFO:teuthology.task.internal:Opening connections... 2023-12-07T16:48:10.115 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-12-07T16:48:10.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:48:10.172 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2023-12-07T16:48:10.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:48:10.223 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-07T16:48:10.232 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-12-07T16:48:10.255 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-12-07T16:48:10.256 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu" 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="20.04" 2023-12-07T16:48:10.305 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-07T16:48:10.306 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-07T16:48:10.306 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-07T16:48:10.306 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-07T16:48:10.306 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=focal 2023-12-07T16:48:10.306 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=focal 2023-12-07T16:48:10.307 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-12-07T16:48:10.330 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-12-07T16:48:10.338 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-12-07T16:48:10.338 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-12-07T16:48:10.386 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Ubuntu" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:ID=ubuntu 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE=debian 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="20.04" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_CODENAME=focal 2023-12-07T16:48:10.387 INFO:teuthology.orchestra.run.smithi184.stdout:UBUNTU_CODENAME=focal 2023-12-07T16:48:10.388 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-12-07T16:48:10.412 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-07T16:48:10.422 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-07T16:48:10.470 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-07T16:48:10.471 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-12-07T16:48:10.473 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2023-12-07T16:48:10.479 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-07T16:48:10.503 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-07T16:48:10.504 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-12-07T16:48:10.508 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2023-12-07T16:48:10.528 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-07T16:48:10.685 INFO:teuthology.run_tasks:Running task kernel... 2023-12-07T16:48:10.705 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-07T16:48:10.705 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2023-12-07T16:48:10.706 DEBUG:teuthology.task.kernel:normalized overrides {'mon.a': {'hwe': True}, 'mon.c': {'hwe': True}, 'mgr.y': {'hwe': True}, 'osd.0': {'hwe': True}, 'osd.1': {'hwe': True}, 'osd.2': {'hwe': True}, 'osd.3': {'hwe': True}, 'client.0': {'hwe': True}, 'ceph.rgw.foo.a': {'hwe': True}, 'node-exporter.a': {'hwe': True}, 'alertmanager.a': {'hwe': True}, 'mon.b': {'hwe': True}, 'mgr.x': {'hwe': True}, 'osd.4': {'hwe': True}, 'osd.5': {'hwe': True}, 'osd.6': {'hwe': True}, 'osd.7': {'hwe': True}, 'client.1': {'hwe': True}, 'prometheus.a': {'hwe': True}, 'grafana.a': {'hwe': True}, 'node-exporter.b': {'hwe': True}, 'ceph.iscsi.iscsi.a': {'hwe': True}} 2023-12-07T16:48:10.706 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro', 'hwe': True}, 'mon.b': {'kdb': True, 'sha1': 'distro', 'hwe': True}}, timeout 300 2023-12-07T16:48:10.706 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-07T16:48:10.707 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-07T16:48:10.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T16:48:10.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T16:48:10.713 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-12-07T16:48:10.714 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-12-07T16:48:10.720 INFO:teuthology.orchestra.run.smithi184.stdout:5.4.0-124-generic 2023-12-07T16:48:10.720 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.4.0-124-generic 2023-12-07T16:48:10.720 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get clean 2023-12-07T16:48:10.721 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-124-generic 2023-12-07T16:48:10.721 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.4.0-124-generic 2023-12-07T16:48:10.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-12-07T16:48:10.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-12-07T16:48:10.822 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get update 2023-12-07T16:48:10.988 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-12-07T16:48:10.988 INFO:teuthology.orchestra.run.smithi184.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-12-07T16:48:10.988 INFO:teuthology.orchestra.run.smithi184.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-12-07T16:48:11.000 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-12-07T16:48:11.000 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-12-07T16:48:11.001 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-12-07T16:48:11.053 INFO:teuthology.orchestra.run.smithi184.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-12-07T16:48:11.076 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-12-07T16:48:11.369 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,604 kB] 2023-12-07T16:48:11.405 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [680 kB] 2023-12-07T16:48:11.496 INFO:teuthology.orchestra.run.smithi184.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,994 kB] 2023-12-07T16:48:11.497 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [680 kB] 2023-12-07T16:48:11.501 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,604 kB] 2023-12-07T16:48:11.505 INFO:teuthology.orchestra.run.smithi184.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [402 kB] 2023-12-07T16:48:11.510 INFO:teuthology.orchestra.run.smithi184.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-12-07T16:48:11.510 INFO:teuthology.orchestra.run.smithi184.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,411 kB] 2023-12-07T16:48:11.538 INFO:teuthology.orchestra.run.smithi184.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [34.7 kB] 2023-12-07T16:48:11.538 INFO:teuthology.orchestra.run.smithi184.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [336 kB] 2023-12-07T16:48:11.541 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [913 kB] 2023-12-07T16:48:11.542 INFO:teuthology.orchestra.run.smithi184.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2023-12-07T16:48:11.542 INFO:teuthology.orchestra.run.smithi184.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [914 kB] 2023-12-07T16:48:11.554 INFO:teuthology.orchestra.run.smithi184.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [633 kB] 2023-12-07T16:48:11.560 INFO:teuthology.orchestra.run.smithi184.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [192 kB] 2023-12-07T16:48:11.562 INFO:teuthology.orchestra.run.smithi184.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2023-12-07T16:48:11.562 INFO:teuthology.orchestra.run.smithi184.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,188 B] 2023-12-07T16:48:11.562 INFO:teuthology.orchestra.run.smithi184.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-12-07T16:48:11.563 INFO:teuthology.orchestra.run.smithi184.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-12-07T16:48:11.563 INFO:teuthology.orchestra.run.smithi184.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-12-07T16:48:11.573 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [402 kB] 2023-12-07T16:48:11.578 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-12-07T16:48:11.578 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [34.7 kB] 2023-12-07T16:48:11.579 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,411 kB] 2023-12-07T16:48:11.596 INFO:teuthology.orchestra.run.smithi184.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [913 kB] 2023-12-07T16:48:11.610 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,994 kB] 2023-12-07T16:48:11.611 INFO:teuthology.orchestra.run.smithi184.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [484 kB] 2023-12-07T16:48:11.611 INFO:teuthology.orchestra.run.smithi184.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2023-12-07T16:48:11.612 INFO:teuthology.orchestra.run.smithi184.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,528 kB] 2023-12-07T16:48:11.622 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [336 kB] 2023-12-07T16:48:11.626 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2023-12-07T16:48:11.626 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [633 kB] 2023-12-07T16:48:11.633 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [914 kB] 2023-12-07T16:48:11.640 INFO:teuthology.orchestra.run.smithi184.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [36.0 kB] 2023-12-07T16:48:11.640 INFO:teuthology.orchestra.run.smithi184.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [353 kB] 2023-12-07T16:48:11.643 INFO:teuthology.orchestra.run.smithi184.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2023-12-07T16:48:11.643 INFO:teuthology.orchestra.run.smithi184.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,139 kB] 2023-12-07T16:48:11.644 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [192 kB] 2023-12-07T16:48:11.645 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2023-12-07T16:48:11.645 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,188 B] 2023-12-07T16:48:11.646 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-12-07T16:48:11.646 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-12-07T16:48:11.650 INFO:teuthology.orchestra.run.smithi184.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [761 kB] 2023-12-07T16:48:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [484 kB] 2023-12-07T16:48:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2023-12-07T16:48:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [36.0 kB] 2023-12-07T16:48:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,528 kB] 2023-12-07T16:48:11.655 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-12-07T16:48:11.656 INFO:teuthology.orchestra.run.smithi184.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [273 kB] 2023-12-07T16:48:11.658 INFO:teuthology.orchestra.run.smithi184.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2023-12-07T16:48:11.658 INFO:teuthology.orchestra.run.smithi184.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2023-12-07T16:48:11.658 INFO:teuthology.orchestra.run.smithi184.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-12-07T16:48:11.663 INFO:teuthology.orchestra.run.smithi184.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-12-07T16:48:11.663 INFO:teuthology.orchestra.run.smithi184.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-12-07T16:48:11.663 INFO:teuthology.orchestra.run.smithi184.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-12-07T16:48:11.663 INFO:teuthology.orchestra.run.smithi184.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-12-07T16:48:11.671 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [353 kB] 2023-12-07T16:48:11.674 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2023-12-07T16:48:11.674 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [761 kB] 2023-12-07T16:48:11.679 INFO:teuthology.orchestra.run.smithi184.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-12-07T16:48:11.680 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,139 kB] 2023-12-07T16:48:11.680 INFO:teuthology.orchestra.run.smithi184.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-12-07T16:48:11.680 INFO:teuthology.orchestra.run.smithi184.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-12-07T16:48:11.680 INFO:teuthology.orchestra.run.smithi184.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-12-07T16:48:11.687 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [273 kB] 2023-12-07T16:48:11.689 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2023-12-07T16:48:11.691 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2023-12-07T16:48:11.691 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-12-07T16:48:11.691 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-12-07T16:48:11.691 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-12-07T16:48:11.707 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-12-07T16:48:11.707 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-12-07T16:48:11.708 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-12-07T16:48:11.708 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-12-07T16:48:11.724 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-12-07T16:48:11.724 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-12-07T16:48:15.495 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 18.3 MB in 3s (6,957 kB/s) 2023-12-07T16:48:15.743 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 18.3 MB in 3s (6,594 kB/s) 2023-12-07T16:48:16.978 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2023-12-07T16:48:17.006 DEBUG:teuthology.orchestra.run.smithi184:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2023-12-07T16:48:17.079 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2023-12-07T16:48:17.255 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-12-07T16:48:17.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2023-12-07T16:48:17.282 INFO:teuthology.orchestra.run.smithi184.stdout:Building dependency tree... 2023-12-07T16:48:17.283 INFO:teuthology.orchestra.run.smithi184.stdout:Reading state information... 2023-12-07T16:48:17.353 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-12-07T16:48:17.461 INFO:teuthology.orchestra.run.smithi184.stdout:The following additional packages will be installed: 2023-12-07T16:48:17.461 INFO:teuthology.orchestra.run.smithi184.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2023-12-07T16:48:17.461 INFO:teuthology.orchestra.run.smithi184.stdout: libusbmuxd6 linux-image-5.15.0-89-generic linux-modules-5.15.0-89-generic 2023-12-07T16:48:17.461 INFO:teuthology.orchestra.run.smithi184.stdout: linux-modules-extra-5.15.0-89-generic thermald upower usbmuxd 2023-12-07T16:48:17.463 INFO:teuthology.orchestra.run.smithi184.stdout:Suggested packages: 2023-12-07T16:48:17.463 INFO:teuthology.orchestra.run.smithi184.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2023-12-07T16:48:17.463 INFO:teuthology.orchestra.run.smithi184.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-89-generic 2023-12-07T16:48:17.520 INFO:teuthology.orchestra.run.smithi184.stdout:The following NEW packages will be installed: 2023-12-07T16:48:17.520 INFO:teuthology.orchestra.run.smithi184.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2023-12-07T16:48:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: libusbmuxd6 linux-image-5.15.0-89-generic linux-image-generic-hwe-20.04 2023-12-07T16:48:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: linux-modules-5.15.0-89-generic linux-modules-extra-5.15.0-89-generic 2023-12-07T16:48:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: thermald upower usbmuxd 2023-12-07T16:48:17.562 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-12-07T16:48:17.563 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-12-07T16:48:17.705 INFO:teuthology.orchestra.run.smithi184.stdout:0 upgraded, 13 newly installed, 0 to remove and 283 not upgraded. 2023-12-07T16:48:17.705 INFO:teuthology.orchestra.run.smithi184.stdout:Need to get 90.8 MB of archives. 2023-12-07T16:48:17.705 INFO:teuthology.orchestra.run.smithi184.stdout:After this operation, 497 MB of additional disk space will be used. 2023-12-07T16:48:17.705 INFO:teuthology.orchestra.run.smithi184.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2023-12-07T16:48:17.763 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-12-07T16:48:17.763 INFO:teuthology.orchestra.run.smithi064.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2023-12-07T16:48:17.763 INFO:teuthology.orchestra.run.smithi064.stdout: libusbmuxd6 linux-image-5.15.0-89-generic linux-modules-5.15.0-89-generic 2023-12-07T16:48:17.763 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-extra-5.15.0-89-generic thermald upower usbmuxd 2023-12-07T16:48:17.764 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-12-07T16:48:17.764 INFO:teuthology.orchestra.run.smithi064.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2023-12-07T16:48:17.764 INFO:teuthology.orchestra.run.smithi064.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-89-generic 2023-12-07T16:48:17.817 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-12-07T16:48:17.817 INFO:teuthology.orchestra.run.smithi064.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2023-12-07T16:48:17.817 INFO:teuthology.orchestra.run.smithi064.stdout: libusbmuxd6 linux-image-5.15.0-89-generic linux-image-generic-hwe-20.04 2023-12-07T16:48:17.817 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-5.15.0-89-generic linux-modules-extra-5.15.0-89-generic 2023-12-07T16:48:17.818 INFO:teuthology.orchestra.run.smithi064.stdout: thermald upower usbmuxd 2023-12-07T16:48:17.875 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 13 newly installed, 0 to remove and 283 not upgraded. 2023-12-07T16:48:17.875 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 90.8 MB of archives. 2023-12-07T16:48:17.875 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 497 MB of additional disk space will be used. 2023-12-07T16:48:17.876 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2023-12-07T16:48:17.933 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2023-12-07T16:48:17.938 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2023-12-07T16:48:17.942 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2023-12-07T16:48:17.959 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2023-12-07T16:48:17.965 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [20.9 MB] 2023-12-07T16:48:17.977 INFO:teuthology.orchestra.run.smithi184.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2023-12-07T16:48:18.006 INFO:teuthology.orchestra.run.smithi184.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2023-12-07T16:48:18.024 INFO:teuthology.orchestra.run.smithi184.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2023-12-07T16:48:18.098 INFO:teuthology.orchestra.run.smithi184.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2023-12-07T16:48:18.134 INFO:teuthology.orchestra.run.smithi184.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [20.9 MB] 2023-12-07T16:48:18.204 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [11.4 MB] 2023-12-07T16:48:18.297 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [58.0 MB] 2023-12-07T16:48:18.966 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.89.99~20.04.47 [2,628 B] 2023-12-07T16:48:19.027 INFO:teuthology.orchestra.run.smithi184.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [11.4 MB] 2023-12-07T16:48:19.042 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2023-12-07T16:48:19.043 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2023-12-07T16:48:19.050 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2023-12-07T16:48:19.051 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2023-12-07T16:48:19.328 INFO:teuthology.orchestra.run.smithi184.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-89-generic amd64 5.15.0-89.99~20.04.1 [58.0 MB] 2023-12-07T16:48:19.745 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 90.8 MB in 1s (74.1 MB/s) 2023-12-07T16:48:19.974 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2023-12-07T16:48:21.245 INFO:teuthology.orchestra.run.smithi184.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.89.99~20.04.47 [2,628 B] 2023-12-07T16:48:21.245 INFO:teuthology.orchestra.run.smithi184.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2023-12-07T16:48:21.246 INFO:teuthology.orchestra.run.smithi184.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2023-12-07T16:48:21.415 INFO:teuthology.orchestra.run.smithi184.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2023-12-07T16:48:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-12-07T16:48:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2023-12-07T16:48:21.767 INFO:teuthology.orchestra.run.smithi184.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2023-12-07T16:48:21.855 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2023-12-07T16:48:22.225 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libplist3:amd64. 2023-12-07T16:48:22.238 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2023-12-07T16:48:22.300 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2023-12-07T16:48:22.516 INFO:teuthology.orchestra.run.smithi184.stdout:Fetched 90.8 MB in 4s (21.3 MB/s) 2023-12-07T16:48:22.661 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2023-12-07T16:48:22.675 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2023-12-07T16:48:22.719 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2023-12-07T16:48:22.732 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2023-12-07T16:48:23.055 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2023-12-07T16:48:23.070 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2023-12-07T16:48:23.115 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2023-12-07T16:48:23.483 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libupower-glib3:amd64. 2023-12-07T16:48:23.499 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2023-12-07T16:48:23.542 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2023-12-07T16:48:23.853 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-5.15.0-89-generic. 2023-12-07T16:48:23.865 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../05-linux-modules-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:23.894 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:24.530 INFO:teuthology.orchestra.run.smithi184.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-12-07T16:48:24.535 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2023-12-07T16:48:24.722 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2023-12-07T16:48:25.066 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libplist3:amd64. 2023-12-07T16:48:25.081 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2023-12-07T16:48:25.166 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2023-12-07T16:48:25.529 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2023-12-07T16:48:25.544 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2023-12-07T16:48:25.586 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2023-12-07T16:48:25.915 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2023-12-07T16:48:25.929 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2023-12-07T16:48:25.981 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2023-12-07T16:48:26.309 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libupower-glib3:amd64. 2023-12-07T16:48:26.324 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2023-12-07T16:48:26.392 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2023-12-07T16:48:26.562 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-5.15.0-89-generic. 2023-12-07T16:48:26.578 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../06-linux-image-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:26.678 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:26.712 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-modules-5.15.0-89-generic. 2023-12-07T16:48:26.725 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../05-linux-modules-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:26.770 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-modules-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-89-generic. 2023-12-07T16:48:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-extra-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:29.254 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-image-5.15.0-89-generic. 2023-12-07T16:48:29.271 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../06-linux-image-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:29.386 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:29.899 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-89-generic. 2023-12-07T16:48:29.915 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-89-generic_5.15.0-89.99~20.04.1_amd64.deb ... 2023-12-07T16:48:29.958 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-modules-extra-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:33.914 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2023-12-07T16:48:33.930 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.89.99~20.04.47_amd64.deb ... 2023-12-07T16:48:33.972 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.89.99~20.04.47) ... 2023-12-07T16:48:34.325 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libevdev2:amd64. 2023-12-07T16:48:34.340 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2023-12-07T16:48:34.383 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2023-12-07T16:48:34.761 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package thermald. 2023-12-07T16:48:34.776 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2023-12-07T16:48:34.820 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2023-12-07T16:48:35.408 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package upower. 2023-12-07T16:48:35.425 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2023-12-07T16:48:35.482 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking upower (0.99.11-1build2) ... 2023-12-07T16:48:35.927 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usbmuxd. 2023-12-07T16:48:35.943 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2023-12-07T16:48:35.987 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2023-12-07T16:48:36.277 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2023-12-07T16:48:36.400 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2023-12-07T16:48:36.534 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2023-12-07T16:48:36.685 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2023-12-07T16:48:36.811 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2023-12-07T16:48:36.852 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2023-12-07T16:48:36.873 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.89.99~20.04.47_amd64.deb ... 2023-12-07T16:48:36.910 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.89.99~20.04.47) ... 2023-12-07T16:48:36.937 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2023-12-07T16:48:37.072 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up upower (0.99.11-1build2) ... 2023-12-07T16:48:37.246 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package libevdev2:amd64. 2023-12-07T16:48:37.263 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2023-12-07T16:48:37.304 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2023-12-07T16:48:37.624 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package thermald. 2023-12-07T16:48:37.635 INFO:teuthology.orchestra.run.smithi064.stdout:upower.service is a disabled or a static unit, not starting it. 2023-12-07T16:48:37.638 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2023-12-07T16:48:37.674 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2023-12-07T16:48:37.709 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2023-12-07T16:48:37.875 INFO:teuthology.orchestra.run.smithi064.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2023-12-07T16:48:37.886 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user `usbmux' (UID 114) ... 2023-12-07T16:48:37.886 INFO:teuthology.orchestra.run.smithi064.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2023-12-07T16:48:38.246 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package upower. 2023-12-07T16:48:38.262 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2023-12-07T16:48:38.319 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking upower (0.99.11-1build2) ... 2023-12-07T16:48:38.409 INFO:teuthology.orchestra.run.smithi064.stdout:Not creating home directory `/var/lib/usbmux'. 2023-12-07T16:48:38.643 INFO:teuthology.orchestra.run.smithi064.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2023-12-07T16:48:38.681 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2023-12-07T16:48:38.742 INFO:teuthology.orchestra.run.smithi184.stdout:Selecting previously unselected package usbmuxd. 2023-12-07T16:48:38.757 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2023-12-07T16:48:38.800 INFO:teuthology.orchestra.run.smithi184.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2023-12-07T16:48:39.010 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2023-12-07T16:48:39.011 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2023-12-07T16:48:39.123 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2023-12-07T16:48:39.230 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2023-12-07T16:48:39.356 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2023-12-07T16:48:39.482 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2023-12-07T16:48:39.484 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:39.608 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2023-12-07T16:48:39.759 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2023-12-07T16:48:39.885 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up upower (0.99.11-1build2) ... 2023-12-07T16:48:40.422 INFO:teuthology.orchestra.run.smithi184.stdout:upower.service is a disabled or a static unit, not starting it. 2023-12-07T16:48:40.472 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2023-12-07T16:48:40.682 INFO:teuthology.orchestra.run.smithi184.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2023-12-07T16:48:40.701 INFO:teuthology.orchestra.run.smithi184.stdout:Adding system user `usbmux' (UID 114) ... 2023-12-07T16:48:40.701 INFO:teuthology.orchestra.run.smithi184.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2023-12-07T16:48:40.850 INFO:teuthology.orchestra.run.smithi064.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:48:40.850 INFO:teuthology.orchestra.run.smithi064.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-89-generic 2023-12-07T16:48:41.021 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-extra-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:41.373 INFO:teuthology.orchestra.run.smithi184.stdout:Not creating home directory `/var/lib/usbmux'. 2023-12-07T16:48:41.576 INFO:teuthology.orchestra.run.smithi184.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2023-12-07T16:48:41.627 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2023-12-07T16:48:41.961 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2023-12-07T16:48:41.961 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2023-12-07T16:48:42.415 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:42.572 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:43.733 INFO:teuthology.orchestra.run.smithi184.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:48:43.733 INFO:teuthology.orchestra.run.smithi184.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-89-generic 2023-12-07T16:48:43.920 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-modules-extra-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:44.040 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.89.99~20.04.47) ... 2023-12-07T16:48:44.165 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-12-07T16:48:44.463 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-12-07T16:48:45.249 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-12-07T16:48:45.298 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-modules-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:45.349 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:45.434 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-12-07T16:48:45.434 INFO:teuthology.orchestra.run.smithi064.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:48:47.530 INFO:teuthology.orchestra.run.smithi184.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.89.99~20.04.47) ... 2023-12-07T16:48:48.615 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-12-07T16:48:48.895 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-12-07T16:48:49.656 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-12-07T16:48:49.757 INFO:teuthology.orchestra.run.smithi184.stdout:Processing triggers for linux-image-5.15.0-89-generic (5.15.0-89.99~20.04.1) ... 2023-12-07T16:48:49.833 INFO:teuthology.orchestra.run.smithi184.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-12-07T16:48:49.833 INFO:teuthology.orchestra.run.smithi184.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:49:11.808 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-12-07T16:49:11.809 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub' 2023-12-07T16:49:11.814 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-07T16:49:11.845 INFO:teuthology.orchestra.run.smithi064.stdout:Generating grub configuration file ... 2023-12-07T16:49:12.168 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:49:12.194 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:49:12.607 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-12-07T16:49:12.689 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-12-07T16:49:14.663 INFO:teuthology.orchestra.run.smithi064.stdout:done 2023-12-07T16:49:15.173 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic-hwe-20.04 2023-12-07T16:49:15.228 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic-hwe-20.04 2023-12-07T16:49:15.228 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-12-07T16:49:15.228 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-12-07T16:49:15.228 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta-hwe-5.15 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.15.0.89.99~20.04.47 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-12-07T16:49:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-12-07T16:49:15.230 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-89-generic 2023-12-07T16:49:15.230 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-07T16:49:15.230 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-07T16:49:15.230 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-07T16:49:15.230 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2023-12-07T16:49:15.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-12-07T16:49:15.281 INFO:teuthology.orchestra.run.smithi184.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-12-07T16:49:15.281 INFO:teuthology.orchestra.run.smithi184.stdout:Sourcing file `/etc/default/grub' 2023-12-07T16:49:15.302 INFO:teuthology.orchestra.run.smithi184.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-07T16:49:15.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-12-07T16:49:15.320 INFO:teuthology.orchestra.run.smithi184.stdout:Generating grub configuration file ... 2023-12-07T16:49:15.454 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-12-07T16:49:15.569 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-12-07T16:49:15.650 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-12-07T16:49:15.686 INFO:teuthology.orchestra.run.smithi184.stdout:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:49:15.710 INFO:teuthology.orchestra.run.smithi184.stdout:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:49:15.732 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-12-07T16:49:16.153 INFO:teuthology.orchestra.run.smithi184.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-12-07T16:49:16.166 INFO:teuthology.orchestra.run.smithi184.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-12-07T16:49:17.247 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-12-07T16:49:17.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2023-12-07T16:49:17.340 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-12-07T16:49:17.547 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-12-07T16:49:17.548 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-12-07T16:49:17.783 INFO:teuthology.orchestra.run.smithi064.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.89.99~20.04.47). 2023-12-07T16:49:17.784 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 0 newly installed, 0 to remove and 283 not upgraded. 2023-12-07T16:49:17.786 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic-hwe-20.04 2023-12-07T16:49:17.809 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic-hwe-20.04 2023-12-07T16:49:17.809 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta-hwe-5.15 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.15.0.89.99~20.04.47 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-07T16:49:17.810 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-12-07T16:49:17.811 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-12-07T16:49:17.811 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-12-07T16:49:17.811 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-12-07T16:49:17.811 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-12-07T16:49:17.858 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.Sm1pGfBSPv 2023-12-07T16:49:17.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Sm1pGfBSPv 2023-12-07T16:49:17.873 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.Sm1pGfBSPv 2023-12-07T16:49:17.917 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.Sm1pGfBSPv is 9KB 2023-12-07T16:49:17.920 INFO:teuthology.orchestra.run.smithi184.stdout:done 2023-12-07T16:49:17.932 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.Sm1pGfBSPv 2023-12-07T16:49:17.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-12-07T16:49:17.998 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T16:49:17.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-12-07T16:49:17.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-12-07T16:49:18.032 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-89-generic 2023-12-07T16:49:18.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo update-grub 2023-12-07T16:49:18.379 DEBUG:teuthology.orchestra.run.smithi184:> dpkg -s linux-image-generic-hwe-20.04 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Package: linux-image-generic-hwe-20.04 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Status: install ok installed 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Priority: optional 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Section: kernel 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Installed-Size: 20 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Maintainer: Ubuntu Kernel Team 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: amd64 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Source: linux-meta-hwe-5.15 2023-12-07T16:49:18.425 INFO:teuthology.orchestra.run.smithi184.stdout:Version: 5.15.0.89.99~20.04.47 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout:Recommends: thermald 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout:Description: Generic Linux kernel image 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout: This package will always depend on the latest generic kernel image 2023-12-07T16:49:18.426 INFO:teuthology.orchestra.run.smithi184.stdout: available. 2023-12-07T16:49:18.426 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-89-generic 2023-12-07T16:49:18.426 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-07T16:49:18.426 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-07T16:49:18.426 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-07T16:49:18.427 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2023-12-07T16:49:18.427 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get clean 2023-12-07T16:49:18.514 DEBUG:teuthology.orchestra.run.smithi184:> sudo apt-get update 2023-12-07T16:49:18.643 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-12-07T16:49:18.644 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-12-07T16:49:18.649 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-12-07T16:49:18.666 INFO:teuthology.orchestra.run.smithi184.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-12-07T16:49:19.570 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub' 2023-12-07T16:49:19.590 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-07T16:49:19.600 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-12-07T16:49:19.888 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:49:19.900 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:49:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-12-07T16:49:20.314 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-12-07T16:49:20.437 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2023-12-07T16:49:20.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2023-12-07T16:49:20.527 INFO:teuthology.orchestra.run.smithi184.stdout:Reading package lists... 2023-12-07T16:49:20.728 INFO:teuthology.orchestra.run.smithi184.stdout:Building dependency tree... 2023-12-07T16:49:20.728 INFO:teuthology.orchestra.run.smithi184.stdout:Reading state information... 2023-12-07T16:49:20.937 INFO:teuthology.orchestra.run.smithi184.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.89.99~20.04.47). 2023-12-07T16:49:20.937 INFO:teuthology.orchestra.run.smithi184.stdout:0 upgraded, 0 newly installed, 0 to remove and 283 not upgraded. 2023-12-07T16:49:20.939 DEBUG:teuthology.orchestra.run.smithi184:> dpkg -s linux-image-generic-hwe-20.04 2023-12-07T16:49:20.955 INFO:teuthology.orchestra.run.smithi184.stdout:Package: linux-image-generic-hwe-20.04 2023-12-07T16:49:20.955 INFO:teuthology.orchestra.run.smithi184.stdout:Status: install ok installed 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Priority: optional 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Section: kernel 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Installed-Size: 20 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Maintainer: Ubuntu Kernel Team 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: amd64 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Source: linux-meta-hwe-5.15 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Version: 5.15.0.89.99~20.04.47 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-89), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Depends: linux-image-5.15.0-89-generic, linux-modules-extra-5.15.0-89-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Recommends: thermald 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout:Description: Generic Linux kernel image 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: This package will always depend on the latest generic kernel image 2023-12-07T16:49:20.956 INFO:teuthology.orchestra.run.smithi184.stdout: available. 2023-12-07T16:49:20.957 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-12-07T16:49:20.962 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.DOvqMcRiN8 2023-12-07T16:49:20.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub/grub.cfg /tmp/tmp.DOvqMcRiN8 2023-12-07T16:49:20.978 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.DOvqMcRiN8 2023-12-07T16:49:21.060 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.DOvqMcRiN8 is 9KB 2023-12-07T16:49:21.114 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.DOvqMcRiN8 2023-12-07T16:49:21.121 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-12-07T16:49:21.136 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-12-07T16:49:21.137 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T16:49:21.137 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-12-07T16:49:21.137 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-12-07T16:49:21.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-12-07T16:49:21.220 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-89-generic 2023-12-07T16:49:21.220 DEBUG:teuthology.orchestra.run.smithi184:> sudo update-grub 2023-12-07T16:49:22.565 INFO:teuthology.orchestra.run.smithi184.stderr:Sourcing file `/etc/default/grub' 2023-12-07T16:49:22.578 INFO:teuthology.orchestra.run.smithi184.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-07T16:49:22.586 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2023-12-07T16:49:22.818 INFO:teuthology.orchestra.run.smithi184.stderr:Found linux image: /boot/vmlinuz-5.15.0-89-generic 2023-12-07T16:49:22.830 INFO:teuthology.orchestra.run.smithi184.stderr:Found initrd image: /boot/initrd.img-5.15.0-89-generic 2023-12-07T16:49:23.166 INFO:teuthology.orchestra.run.smithi184.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-12-07T16:49:23.178 INFO:teuthology.orchestra.run.smithi184.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-12-07T16:49:23.958 INFO:teuthology.orchestra.run.smithi184.stderr:done 2023-12-07T16:49:23.959 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2023-12-07T16:49:51.170 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-07T16:49:51.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T16:49:51.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:49:53.965 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-07T16:49:53.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T16:49:53.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:50:25.763 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-12-07T16:50:34.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T16:50:34.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:50:37.833 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-12-07T16:50:49.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T16:50:49.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:50:52.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-12-07T16:50:53.969 DEBUG:teuthology.orchestra.remote:timed out 2023-12-07T16:51:02.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T16:51:02.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:51:02.985 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-12-07T16:51:07.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T16:51:07.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:51:08.186 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-12-07T16:51:08.794 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T16:51:08.794 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-89-generic"... 2023-12-07T16:51:08.794 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-12-07T16:51:08.844 INFO:teuthology.orchestra.run.smithi064.stdout:5.15.0-89-generic 2023-12-07T16:51:08.844 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-89-generic vs 5.15.0-89-generic 2023-12-07T16:51:08.844 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-07T16:51:08.844 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-07T16:51:09.850 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-07T16:51:09.850 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-07T16:51:09.944 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2023-12-07T16:51:09.976 DEBUG:teuthology.parallel:result is None 2023-12-07T16:51:14.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T16:51:14.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T16:51:15.223 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-12-07T16:51:15.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T16:51:15.737 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-89-generic"... 2023-12-07T16:51:15.737 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-12-07T16:51:15.787 INFO:teuthology.orchestra.run.smithi184.stdout:5.15.0-89-generic 2023-12-07T16:51:15.787 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-89-generic vs 5.15.0-89-generic 2023-12-07T16:51:15.787 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-07T16:51:15.787 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-07T16:51:16.788 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-07T16:51:16.788 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-07T16:51:17.133 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2023-12-07T16:51:17.175 DEBUG:teuthology.parallel:result is None 2023-12-07T16:51:17.176 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-07T16:51:17.186 INFO:teuthology.task.internal:Creating test directory... 2023-12-07T16:51:17.186 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-07T16:51:17.189 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-07T16:51:17.195 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-07T16:51:17.237 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-07T16:51:17.370 INFO:teuthology.task.internal:Creating archive directory... 2023-12-07T16:51:17.371 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-07T16:51:17.373 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-07T16:51:17.414 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-07T16:51:17.446 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-07T16:51:17.446 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-07T16:51:17.449 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-07T16:51:17.465 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T16:51:17.465 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T16:51:17.472 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T16:51:17.474 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T16:51:17.476 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-07T16:51:17.557 INFO:teuthology.task.internal:Configuring sudo... 2023-12-07T16:51:17.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-07T16:51:17.560 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-07T16:51:17.575 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-07T16:51:17.622 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-07T16:51:17.622 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-07T16:51:17.637 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-07T16:51:17.645 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T16:51:17.691 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T16:51:17.740 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T16:51:17.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-07T16:51:17.809 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T16:51:17.818 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T16:51:17.868 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T16:51:17.868 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-07T16:51:17.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-12-07T16:51:17.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2023-12-07T16:51:18.046 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-07T16:51:18.054 INFO:teuthology.task.internal:Starting timer... 2023-12-07T16:51:18.054 INFO:teuthology.run_tasks:Running task pcp... 2023-12-07T16:51:18.112 INFO:teuthology.run_tasks:Running task selinux... 2023-12-07T16:51:18.123 DEBUG:teuthology.task.selinux:Excluding smithi064: OS 'ubuntu' does not support SELinux 2023-12-07T16:51:18.123 DEBUG:teuthology.task.selinux:Excluding smithi184: OS 'ubuntu' does not support SELinux 2023-12-07T16:51:18.123 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-07T16:51:18.123 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-12-07T16:51:18.124 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-07T16:51:18.124 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-07T16:51:18.219 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-07T16:51:18.219 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-07T16:51:18.252 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-07T16:51:18.253 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-07T17:02:51.384 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2023-12-07T17:02:51.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T17:02:51.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T17:02:51.480 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-12-07T17:02:51.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-12-07T17:02:51.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T17:02:51.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T17:02:51.689 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-12-07T17:02:51.811 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-12-07T17:02:51.811 INFO:teuthology.run_tasks:Running task clock... 2023-12-07T17:02:51.827 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-07T17:02:51.828 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-07T17:02:51.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T17:02:51.831 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-07T17:02:51.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T17:02:51.862 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-12-07T17:02:51.862 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Command line: ntpd -gq 2023-12-07T17:02:51.863 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: proto: precision = 0.080 usec (-23) 2023-12-07T17:02:51.863 INFO:teuthology.orchestra.run.smithi064.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-07T17:02:51.863 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-07T17:02:51.864 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: restrict ::: KOD does nothing without LIMITED. 2023-12-07T17:02:51.864 INFO:teuthology.orchestra.run.smithi064.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen and drop on 0 v6wildcard [::]:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen normally on 2 lo 127.0.0.1:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen normally on 3 enp3s0f1 172.21.15.64:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen normally on 4 lo [::1]:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8df%5]:123 2023-12-07T17:02:51.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:51 ntpd[23467]: Listening on routing socket on fd #22 for interface updates 2023-12-07T17:02:51.883 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-12-07T17:02:51.883 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Command line: ntpd -gq 2023-12-07T17:02:51.884 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: proto: precision = 0.074 usec (-24) 2023-12-07T17:02:51.884 INFO:teuthology.orchestra.run.smithi184.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-07T17:02:51.884 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-07T17:02:51.885 INFO:teuthology.orchestra.run.smithi184.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: restrict ::: KOD does nothing without LIMITED. 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen and drop on 0 v6wildcard [::]:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen normally on 2 lo 127.0.0.1:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen normally on 3 enp3s0f1 172.21.15.184:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen normally on 4 lo [::1]:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7b8f%5]:123 2023-12-07T17:02:51.886 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:51 ntpd[23478]: Listening on routing socket on fd #22 for interface updates 2023-12-07T17:02:52.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:52.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:52.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:52.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:52.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:52.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:52 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:52.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:52.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:52.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:52.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:52.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:52.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:52 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:53.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:53.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:53.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:53.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:53.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:53.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:53 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:54.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:54.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:54 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:54.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:54.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:54 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:55.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:55.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:55.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:55.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:55.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:55.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:55.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:55.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:55 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:55.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:55.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:55.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:55.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:55.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:55.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:55 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:56.865 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:56 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:56.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:56.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:56.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:56.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:56.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:56 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:57.863 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:57 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:57.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:57.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:57.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:57.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:57.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:57.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:57 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:58.864 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:58.865 INFO:teuthology.orchestra.run.smithi064.stdout: 7 Dec 17:02:58 ntpd[23467]: ntpd: time slew +0.002342 s 2023-12-07T17:02:58.865 INFO:teuthology.orchestra.run.smithi064.stdout:ntpd: time slew +0.002342s 2023-12-07T17:02:58.865 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-07T17:02:58.865 INFO:teuthology.orchestra.run.smithi064.stderr: 7 Dec 17:02:58 ntpd[23467]: can't open /var/log/ntpstats/loopstats.20231207: Permission denied 2023-12-07T17:02:58.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.884 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: can't open /var/log/ntpstats/rawstats.20231207: Permission denied 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: can't open /var/log/ntpstats/peerstats.20231207: Permission denied 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: 7 Dec 17:02:58 ntpd[23478]: ntpd: time slew +0.000449 s 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stdout:ntpd: time slew +0.000449s 2023-12-07T17:02:58.885 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-07T17:02:58.886 INFO:teuthology.orchestra.run.smithi184.stderr: 7 Dec 17:02:58 ntpd[23478]: can't open /var/log/ntpstats/loopstats.20231207: Permission denied 2023-12-07T17:02:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-12-07T17:02:58.922 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-12-07T17:02:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.942 INFO:teuthology.orchestra.run.smithi184.stdout: remote refid st t when poll reach delay offset jitter 2023-12-07T17:02:58.942 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================== 2023-12-07T17:02:58.942 INFO:teuthology.orchestra.run.smithi184.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.942 INFO:teuthology.orchestra.run.smithi184.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.943 INFO:teuthology.orchestra.run.smithi184.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.943 INFO:teuthology.orchestra.run.smithi184.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-12-07T17:02:58.944 INFO:teuthology.run_tasks:Running task nvme_loop... 2023-12-07T17:02:58.957 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2023-12-07T17:02:58.957 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:02:58.958 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:02:58.970 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-07T17:02:58.970 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-12-07T17:02:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-07T17:02:59.018 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:02:59.018 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 490 Links: 1 2023-12-07T17:02:59.019 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:02:59.019 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:01:19.273118335 +0000 2023-12-07T17:02:59.019 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:01:18.753130181 +0000 2023-12-07T17:02:59.019 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:01:18.753130181 +0000 2023-12-07T17:02:59.019 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:02:59.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-07T17:02:59.074 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:02:59.074 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:02:59.074 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000400151 s, 1.3 MB/s 2023-12-07T17:02:59.075 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-07T17:02:59.128 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-12-07T17:02:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-07T17:02:59.182 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 496 Links: 1 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:01:20.149098378 +0000 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:01:19.477113688 +0000 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:01:19.477113688 +0000 2023-12-07T17:02:59.183 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:02:59.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-07T17:02:59.236 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:02:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:02:59.237 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000405877 s, 1.3 MB/s 2023-12-07T17:02:59.238 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-07T17:02:59.288 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 501 Links: 1 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:01:20.869081975 +0000 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:01:20.381093092 +0000 2023-12-07T17:02:59.338 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:01:20.381093092 +0000 2023-12-07T17:02:59.339 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:02:59.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-07T17:02:59.394 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:02:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:02:59.395 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000405046 s, 1.3 MB/s 2023-12-07T17:02:59.395 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-07T17:02:59.448 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 507 Links: 1 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:01:21.509067393 +0000 2023-12-07T17:02:59.498 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:01:21.081077146 +0000 2023-12-07T17:02:59.499 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:01:21.081077146 +0000 2023-12-07T17:02:59.499 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:02:59.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-07T17:02:59.553 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:02:59.554 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:02:59.554 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000414241 s, 1.2 MB/s 2023-12-07T17:02:59.555 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-07T17:02:59.607 DEBUG:teuthology.orchestra.run.smithi064:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2023-12-07T17:02:59.739 INFO:teuthology.orchestra.run.smithi064.stdout:loop 2023-12-07T17:02:59.741 INFO:tasks.nvme_loop:Connecting nvme_loop smithi064:/dev/vg_nvme/lv_1... 2023-12-07T17:02:59.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2023-12-07T17:02:59.761 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-12-07T17:02:59.788 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/vg_nvme/lv_11 2023-12-07T17:02:59.810 INFO:tasks.nvme_loop:Connecting nvme_loop smithi064:/dev/vg_nvme/lv_2... 2023-12-07T17:02:59.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2023-12-07T17:02:59.880 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-12-07T17:02:59.905 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/vg_nvme/lv_21 2023-12-07T17:02:59.931 INFO:tasks.nvme_loop:Connecting nvme_loop smithi064:/dev/vg_nvme/lv_3... 2023-12-07T17:02:59.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2023-12-07T17:03:00.005 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-12-07T17:03:00.031 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/vg_nvme/lv_31 2023-12-07T17:03:00.055 INFO:tasks.nvme_loop:Connecting nvme_loop smithi064:/dev/vg_nvme/lv_4... 2023-12-07T17:03:00.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2023-12-07T17:03:00.121 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-12-07T17:03:00.146 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/vg_nvme/lv_41 2023-12-07T17:03:00.170 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:03:00.170 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:03:00.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo nvme list 2023-12-07T17:03:00.277 INFO:teuthology.orchestra.run.smithi064.stdout:Node SN Model Namespace Usage Format FW Rev 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/nvme0n1 CVFT534400C2400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/nvme1n1 ccaaa05545f752182028 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/nvme2n1 5e8329442eff049bc7eb Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/nvme3n1 378bad0c1bc115e6b68e Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:00.278 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/nvme4n1 4b8306b6109d12323fb8 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:00.279 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-12-07T17:03:00.279 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:03:00.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/scratch_devs 2023-12-07T17:03:00.338 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:03:00.338 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:03:00.345 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-07T17:03:00.346 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 490 Links: 1 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:01:16.153025130 +0000 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:01:15.609037525 +0000 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:01:15.609037525 +0000 2023-12-07T17:03:00.396 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:03:00.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-07T17:03:00.455 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:03:00.455 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:03:00.455 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000429971 s, 1.2 MB/s 2023-12-07T17:03:00.456 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-07T17:03:00.509 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 496 Links: 1 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:01:16.881008541 +0000 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:01:16.357020481 +0000 2023-12-07T17:03:00.559 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:01:16.357020481 +0000 2023-12-07T17:03:00.560 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:03:00.562 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-07T17:03:00.615 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:03:00.615 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:03:00.615 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000405761 s, 1.3 MB/s 2023-12-07T17:03:00.616 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-07T17:03:00.669 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 503 Links: 1 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:01:17.632991405 +0000 2023-12-07T17:03:00.725 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:01:17.093003710 +0000 2023-12-07T17:03:00.726 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:01:17.093003710 +0000 2023-12-07T17:03:00.726 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:03:00.726 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-07T17:03:00.783 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:03:00.783 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:03:00.783 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000417094 s, 1.2 MB/s 2023-12-07T17:03:00.784 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-07T17:03:00.841 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 509 Links: 1 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:01:18.392974087 +0000 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:01:17.852986392 +0000 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:01:17.852986392 +0000 2023-12-07T17:03:00.892 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:03:00.893 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-07T17:03:00.951 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:03:00.951 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:03:00.951 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000403348 s, 1.3 MB/s 2023-12-07T17:03:00.952 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-07T17:03:01.005 DEBUG:teuthology.orchestra.run.smithi184:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2023-12-07T17:03:01.129 INFO:teuthology.orchestra.run.smithi184.stdout:loop 2023-12-07T17:03:01.131 INFO:tasks.nvme_loop:Connecting nvme_loop smithi184:/dev/vg_nvme/lv_1... 2023-12-07T17:03:01.131 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2023-12-07T17:03:01.149 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-12-07T17:03:01.176 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/vg_nvme/lv_11 2023-12-07T17:03:01.199 INFO:tasks.nvme_loop:Connecting nvme_loop smithi184:/dev/vg_nvme/lv_2... 2023-12-07T17:03:01.199 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2023-12-07T17:03:01.267 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-12-07T17:03:01.294 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/vg_nvme/lv_21 2023-12-07T17:03:01.317 INFO:tasks.nvme_loop:Connecting nvme_loop smithi184:/dev/vg_nvme/lv_3... 2023-12-07T17:03:01.317 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2023-12-07T17:03:01.385 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-12-07T17:03:01.409 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/vg_nvme/lv_31 2023-12-07T17:03:01.433 INFO:tasks.nvme_loop:Connecting nvme_loop smithi184:/dev/vg_nvme/lv_4... 2023-12-07T17:03:01.433 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2023-12-07T17:03:01.504 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-12-07T17:03:01.529 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/vg_nvme/lv_41 2023-12-07T17:03:01.553 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:03:01.553 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:03:01.603 DEBUG:teuthology.orchestra.run.smithi184:> sudo nvme list 2023-12-07T17:03:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:Node SN Model Namespace Usage Format FW Rev 2023-12-07T17:03:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2023-12-07T17:03:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/nvme0n1 CVFT626500CE400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2023-12-07T17:03:01.659 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/nvme1n1 b932284d98071b791921 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:01.660 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/nvme2n1 4926ec231b9ac9083dfd Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:01.660 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/nvme3n1 9b147fc95727322cef33 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:01.660 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/nvme4n1 3d9b812c2e8846372c30 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-8 2023-12-07T17:03:01.661 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-12-07T17:03:01.661 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:03:01.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/scratch_devs 2023-12-07T17:03:01.722 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-07T17:03:01.808 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'e068ebcf7ecc6503f24666fb6b152034d3fe1067'} 2023-12-07T17:03:01.809 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T17:03:01.809 INFO:tasks.cephadm:Cluster fsid is 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:03:01.809 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-07T17:03:01.809 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.c': '[v2:172.21.15.64:3301,v1:172.21.15.64:6790]', 'mon.b': '172.21.15.184'} 2023-12-07T17:03:01.809 INFO:tasks.cephadm:First mon is mon.a on smithi064 2023-12-07T17:03:01.809 INFO:tasks.cephadm:First mgr is y 2023-12-07T17:03:01.810 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-07T17:03:01.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2023-12-07T17:03:01.827 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2023-12-07T17:03:01.843 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-07T17:03:01.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T17:03:02.064 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e068ebcf7ecc6503f24666fb6b152034d3fe1067', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=focal,DIST=focal,MACHINE_SIZE=gigantic/74753/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-64-ge068ebcf', 'node_name': '172.21.5.38+adami08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=focal,DIST=focal,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-64-ge068ebcf-1focal'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-06-1425/e068ebcf7ecc6503f24666fb6b152034d3fe1067/ubuntu/focal/flavors/default/', 'distro_codename': 'focal', 'modified': '2023-12-07 00:59:41.089290', 'distro_version': '20.04', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri8-testing-2023-12-06-1425', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-yuri8-testing-2023-12-06-1425/e068ebcf7ecc6503f24666fb6b152034d3fe1067/ubuntu/focal/flavors/default/', 'archs': ['x86_64'], 'distro': 'ubuntu'}] 2023-12-07T17:03:02.204 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-yuri8-testing-2023-12-06-1425, sha1 e068ebcf7ecc6503f24666fb6b152034d3fe1067 from https://shaman.ceph.com/api/search/?project=ceph&distros=ubuntu%2F20.04%2Fx86_64&flavor=default&sha1=e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T17:03:02.204 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-yuri8-testing-2023-12-06-1425/e068ebcf7ecc6503f24666fb6b152034d3fe1067/ubuntu/focal/x86_64/flavors/default/cephadm 2023-12-07T17:03:02.205 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-yuri8-testing-2023-12-06-1425/e068ebcf7ecc6503f24666fb6b152034d3fe1067/ubuntu/focal/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.046 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r-- 1 ubuntu ubuntu 832268 Dec 7 17:03 /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.046 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-yuri8-testing-2023-12-06-1425/e068ebcf7ecc6503f24666fb6b152034d3fe1067/ubuntu/focal/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.927 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-rw-r-- 1 ubuntu ubuntu 832268 Dec 7 17:03 /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.927 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.937 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-07T17:03:03.957 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 on all hosts... 2023-12-07T17:03:03.957 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 pull 2023-12-07T17:03:03.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 pull 2023-12-07T17:03:04.227 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067... 2023-12-07T17:03:04.239 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067... 2023-12-07T17:03:58.125 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev)", 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "e71bc55fd8bb7312113b93709199b3046fb27a1249ab2ff38e9d4fb4046d0412", 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f77e76429b2f90f995fd0b15e8ce2605700ed2a51f2d75623215c9aaf74f90b6" 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-12-07T17:03:58.126 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-12-07T17:04:00.602 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev)", 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "e71bc55fd8bb7312113b93709199b3046fb27a1249ab2ff38e9d4fb4046d0412", 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f77e76429b2f90f995fd0b15e8ce2605700ed2a51f2d75623215c9aaf74f90b6" 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2023-12-07T17:04:00.603 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-12-07T17:04:00.633 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2023-12-07T17:04:00.649 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2023-12-07T17:04:00.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2023-12-07T17:04:00.709 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2023-12-07T17:04:00.723 INFO:tasks.cephadm:Writing seed config... 2023-12-07T17:04:00.724 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-07T17:04:00.724 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-07T17:04:00.725 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-07T17:04:00.726 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:04:00.726 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-07T17:04:00.760 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7ae18f76-9522-11ee-95a2-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-12-07T17:04:00.760 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service 2023-12-07T17:04:00.803 DEBUG:teuthology.orchestra.run.smithi064:mgr.y> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y.service 2023-12-07T17:04:00.846 INFO:tasks.cephadm:Bootstrapping... 2023-12-07T17:04:00.847 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 -v bootstrap --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:04:01.054 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-12-07T17:04:01.054 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067', '-v', 'bootstrap', '--fsid', '7ae18f76-9522-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2023-12-07T17:04:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-07T17:04:01.055 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2023-12-07T17:04:01.055 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2023-12-07T17:04:01.055 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2023-12-07T17:04:01.061 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-07T17:04:01.061 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-07T17:04:01.067 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-07T17:04:01.067 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.072 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2023-12-07T17:04:01.072 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2023-12-07T17:04:01.078 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2023-12-07T17:04:01.078 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.083 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2023-12-07T17:04:01.083 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout masked 2023-12-07T17:04:01.088 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2023-12-07T17:04:01.088 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2023-12-07T17:04:01.094 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2023-12-07T17:04:01.099 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2023-12-07T17:04:01.099 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.105 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2023-12-07T17:04:01.111 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2023-12-07T17:04:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:Unit ntp.service is enabled and running 2023-12-07T17:04:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2023-12-07T17:04:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:docker (/usr/bin/docker) is present 2023-12-07T17:04:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2023-12-07T17:04:01.112 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2023-12-07T17:04:01.117 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-07T17:04:01.118 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-07T17:04:01.124 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-07T17:04:01.124 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.130 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2023-12-07T17:04:01.131 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2023-12-07T17:04:01.136 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2023-12-07T17:04:01.136 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.142 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2023-12-07T17:04:01.142 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout masked 2023-12-07T17:04:01.147 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2023-12-07T17:04:01.147 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.153 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2023-12-07T17:04:01.153 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2023-12-07T17:04:01.160 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2023-12-07T17:04:01.160 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-12-07T17:04:01.166 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2023-12-07T17:04:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:04:01.171 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Unit ntp.service is enabled and running 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140227297098480 on /run/cephadm/7ae18f76-9522-11ee-95a2-87774f69a715.lock 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140227297098480 acquired on /run/cephadm/7ae18f76-9522-11ee-95a2-87774f69a715.lock 2023-12-07T17:04:01.172 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2023-12-07T17:04:01.173 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2023-12-07T17:04:01.173 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2023-12-07T17:04:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2023-12-07T17:04:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2023-12-07T17:04:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2023-12-07T17:04:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 2023-12-07T17:04:01.179 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.64 metric 100 2023-12-07T17:04:01.183 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-07T17:04:01.183 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2023-12-07T17:04:01.183 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1764sec pref medium 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-12-07T17:04:01.187 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-07T17:04:01.187 INFO:journalctl@ceph.mgr.y.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:04:01.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2023-12-07T17:04:01.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2023-12-07T17:04:01.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.15.254/32` 2023-12-07T17:04:01.188 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.15.254/32` 2023-12-07T17:04:01.188 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.15.254/32', '172.21.15.254/32'] 2023-12-07T17:04:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-07T17:04:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067... 2023-12-07T17:04:01.423 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/docker: stdout e068ebcf7ecc6503f24666fb6b152034d3fe1067: Pulling from ceph-ci/ceph 2023-12-07T17:04:01.423 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/docker: stdout Digest: sha256:f77e76429b2f90f995fd0b15e8ce2605700ed2a51f2d75623215c9aaf74f90b6 2023-12-07T17:04:01.423 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T17:04:01.423 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/docker: stdout quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 2023-12-07T17:04:02.979 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:04:02.979 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:04:02.979 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2023-12-07T17:04:04.409 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2023-12-07T17:04:04.409 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2023-12-07T17:04:05.791 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAF+3Flw6KFDxAAafSs4iLsofRm6BbjTZuz6w== 2023-12-07T17:04:07.111 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAG+3Fl41GYIhAAo89ZJUAABgQqkaDGTP6kag== 2023-12-07T17:04:08.418 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQAH+3FlmHwGNRAAwPBrU6JB2SCeAcLkHJBQ4w== 2023-12-07T17:04:08.418 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = pacific 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-07T17:04:09.845 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:04:09.846 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 0 set uid:gid to 167:167 (ceph:ceph) 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 1 imported monmap: 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr epoch 0 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr fsid 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr last_changed 2023-12-07T17:04:09.272348+0000 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr created 2023-12-07T17:04:09.272348+0000 2023-12-07T17:04:11.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr min_mon_release 16 (pacific) 2023-12-07T17:04:11.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2023-12-07T17:04:11.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2023-12-07T17:04:11.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 0 /usr/bin/ceph-mon: set fsid to 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: RocksDB version: 7.9.2 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Git sha 0 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Compile date 2023-12-06 23:35:10 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: DB SUMMARY 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: DB Session ID: HDQ07BQH2LA3P9C9791S 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.375 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.error_if_exists: 0 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.create_if_missing: 1 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.paranoid_checks: 1 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.flush_verify_memtable_count: 1 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.env: 0x55ef26977a60 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.fs: PosixFileSystem 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.info_log: 0x55ef28c889c0 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_file_opening_threads: 16 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.statistics: (nil) 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.use_fsync: 0 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_log_file_size: 0 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-07T17:04:11.376 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.log_file_time_to_roll: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.keep_log_file_num: 1000 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.recycle_log_file_num: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_fallocate: 1 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_mmap_reads: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_mmap_writes: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.use_direct_reads: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.create_missing_column_families: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.db_log_dir: 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.wal_dir: 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.table_cache_numshardbits: 6 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.WAL_ttl_seconds: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.WAL_size_limit_MB: 0 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-07T17:04:11.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.is_fd_close_on_exec: 1 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.advise_random_on_open: 1 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.db_write_buffer_size: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.write_buffer_manager: 0x55ef28c5d220 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.use_adaptive_mutex: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.rate_limiter: (nil) 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.wal_recovery_mode: 2 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enable_thread_tracking: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enable_pipelined_write: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.unordered_write: 0 2023-12-07T17:04:11.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.row_cache: None 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.wal_filter: None 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_ingest_behind: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.two_write_queues: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.manual_wal_flush: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.wal_compression: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.atomic_flush: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.persist_stats_to_disk: 0 2023-12-07T17:04:11.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.write_dbid_to_manifest: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.log_readahead_size: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.best_efforts_recovery: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.allow_data_in_errors: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.db_host_id: __hostname__ 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enforce_single_del_contracts: true 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_background_jobs: 2 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_background_compactions: -1 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_subcompactions: 1 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.delayed_write_rate : 16777216 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_total_wal_size: 0 2023-12-07T17:04:11.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.stats_dump_period_sec: 600 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.stats_persist_period_sec: 600 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_open_files: -1 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bytes_per_sync: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.wal_bytes_per_sync: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.strict_bytes_per_sync: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_readahead_size: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_background_flushes: -1 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Compression algorithms supported: 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kZSTD supported: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kXpressCompression supported: 0 2023-12-07T17:04:11.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kLZ4HCCompression supported: 1 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kZlibCompression supported: 1 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kSnappyCompression supported: 1 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kLZ4Compression supported: 1 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: kBZip2Compression supported: 0 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Fast CRC32 supported: Supported on x86 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: DMutex implementation: pthread_mutex_t 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-07T17:04:11.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.merge_operator: 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_filter: None 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_filter_factory: None 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.sst_partitioner_factory: None 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.memtable_factory: SkipListFactory 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.table_factory: BlockBasedTable 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ef28c88ac0) 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2023-12-07T17:04:11.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr checksum: 4 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x55ef28cdf090 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2023-12-07T17:04:11.384 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr filter_policy: bloomfilter 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr format_version: 5 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr max_auto_readahead_size: 262144 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr prepopulate_block_cache: 0 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr initial_auto_readahead_size: 8192 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr num_file_reads_for_auto_readahead: 2 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.write_buffer_size: 33554432 2023-12-07T17:04:11.385 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_write_buffer_number: 2 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression: NoCompression 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression: Disabled 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.prefix_extractor: nullptr 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.num_levels: 7 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-07T17:04:11.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.window_bits: -14 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.level: 32767 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.strategy: 0 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.enabled: false 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-07T17:04:11.387 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.target_file_size_base: 67108864 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.target_file_size_multiplier: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-07T17:04:11.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.arena_block_size: 1048576 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.disable_auto_compactions: 0 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.table_properties_collectors: 2023-12-07T17:04:11.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.inplace_update_support: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.inplace_update_num_locks: 10000 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.memtable_huge_page_size: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.bloom_locality: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.max_successive_merges: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.optimize_filters_for_hits: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.paranoid_file_checks: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.force_consistency_checks: 1 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.report_bg_io_stats: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.ttl: 2592000 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.periodic_compaction_seconds: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-07T17:04:11.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enable_blob_files: false 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.min_blob_size: 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_file_size: 268435456 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_compression_type: NoCompression 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.enable_blob_garbage_collection: false 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.blob_file_starting_level: 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.679+0000 7fb31f13ac80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 31bb0bed-33a5-44c1-be3c-a3f6d48e82a3 2023-12-07T17:04:11.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ef28d50000 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb31f13ac80 4 rocksdb: DB pointer 0x55ef28d42000 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb30f2f3700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.683+0000 7fb30f2f3700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T17:04:11.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T17:04:11.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Block cache BinnedLRUCache@0x55ef28cdf090#7 capacity: 512.00 MB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.687+0000 7fb31f13ac80 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.687+0000 7fb31f13ac80 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2023-12-07T17:04:11.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-mon: stderr debug 2023-12-07T17:04:10.687+0000 7fb31f13ac80 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2023-12-07T17:04:11.395 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2023-12-07T17:04:11.798 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2023-12-07T17:04:12.019 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target -> /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target. 2023-12-07T17:04:12.019 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target -> /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target. 2023-12-07T17:04:12.308 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a 2023-12-07T17:04:12.308 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-12-07T17:04:12.506 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target.wants/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@.service. 2023-12-07T17:04:12.517 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld does not appear to be present 2023-12-07T17:04:12.517 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2023-12-07T17:04:12.517 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2023-12-07T17:04:12.517 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2023-12-07T17:04:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:13 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.878456+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T17:04:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2023-12-07T17:04:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-07T17:04:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2023-12-07T17:04:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.354359s) 2023-12-07T17:04:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-07T17:04:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2023-12-07T17:04:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2023-12-07T17:04:15.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.884343+0000 mon.a (mon.0) 2 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T17:04:15.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.884600+0000 mon.a (mon.0) 3 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:04:15.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.888139+0000 mon.a (mon.0) 4 : cluster 0 fsmap 2023-12-07T17:04:15.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.890440+0000 mon.a (mon.0) 5 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2023-12-07T17:04:15.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: cluster 2023-12-07T17:04:13.890906+0000 mon.a (mon.0) 6 : cluster 0 mgrmap e1: no daemons active 2023-12-07T17:04:15.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:14 smithi064 bash[24531]: audit 2023-12-07T17:04:14.238630+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.64:0/1716806458' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-07T17:04:16.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:15 smithi064 bash[24531]: audit 2023-12-07T17:04:15.881844+0000 mon.a (mon.0) 8 : audit 1 from='client.? 172.21.15.64:0/3930225926' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-07T17:04:16.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:15 smithi064 bash[24531]: audit 2023-12-07T17:04:15.886791+0000 mon.a (mon.0) 9 : audit 1 from='client.? 172.21.15.64:0/3930225926' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-07T17:04:16.527 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-07T17:04:16.528 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2023-12-07T17:04:18.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:17 smithi064 bash[24531]: audit 2023-12-07T17:04:17.627726+0000 mon.a (mon.0) 10 : audit 0 from='client.? 172.21.15.64:0/254651514' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:04:18.213 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2023-12-07T17:04:18.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:18 smithi064 systemd[1]: Stopping Ceph mon.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:04:18.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:18 smithi064 bash[24531]: debug 2023-12-07T17:04:18.383+0000 7fabf1dd6700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-07T17:04:18.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:18 smithi064 bash[24531]: debug 2023-12-07T17:04:18.383+0000 7fabf1dd6700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-07T17:04:19.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:18 smithi064 bash[24938]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-mon-a 2023-12-07T17:04:19.155 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.15.254/32,172.21.0.0/20 in mon config section 2023-12-07T17:04:19.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:19 smithi064 bash[25016]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-mon-a 2023-12-07T17:04:19.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:19 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-12-07T17:04:19.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:19 smithi064 systemd[1]: Stopped Ceph mon.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:04:19.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:19 smithi064 systemd[1]: Started Ceph mon.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:04:20.612 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.455+0000 7f1b1817ec80 0 set uid:gid to 167:167 (ceph:ceph) 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.455+0000 7f1b1817ec80 0 ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev), process ceph-mon, pid 7 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.455+0000 7f1b1817ec80 0 pidfile_write: ignore empty --pid-file 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.463+0000 7f1b1817ec80 0 load: jerasure load: lrc 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: RocksDB version: 7.9.2 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Git sha 0 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Compile date 2023-12-06 23:35:10 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: DB SUMMARY 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: DB Session ID: VZG8BM7R4X3ZHGEVZ3TV 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: CURRENT file: CURRENT 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: IDENTITY file: IDENTITY 2023-12-07T17:04:20.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83772 ; 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.error_if_exists: 0 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.create_if_missing: 0 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.paranoid_checks: 1 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.flush_verify_memtable_count: 1 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.env: 0x5574acf50a60 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.fs: PosixFileSystem 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.info_log: 0x5574ae8bf780 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_file_opening_threads: 16 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.statistics: (nil) 2023-12-07T17:04:20.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.use_fsync: 0 2023-12-07T17:04:20.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_log_file_size: 0 2023-12-07T17:04:20.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.log_file_time_to_roll: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.keep_log_file_num: 1000 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.recycle_log_file_num: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_fallocate: 1 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_mmap_reads: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_mmap_writes: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.use_direct_reads: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.create_missing_column_families: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.db_log_dir: 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.wal_dir: 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.table_cache_numshardbits: 6 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.WAL_ttl_seconds: 0 2023-12-07T17:04:20.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.WAL_size_limit_MB: 0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.is_fd_close_on_exec: 1 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.advise_random_on_open: 1 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.db_write_buffer_size: 0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.write_buffer_manager: 0x5574ae8932c0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.use_adaptive_mutex: 0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.rate_limiter: (nil) 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.wal_recovery_mode: 2 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enable_thread_tracking: 0 2023-12-07T17:04:20.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enable_pipelined_write: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.unordered_write: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.row_cache: None 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.wal_filter: None 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_ingest_behind: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.two_write_queues: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.manual_wal_flush: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.wal_compression: 0 2023-12-07T17:04:20.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.atomic_flush: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.persist_stats_to_disk: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.write_dbid_to_manifest: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.log_readahead_size: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.best_efforts_recovery: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.allow_data_in_errors: 0 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.db_host_id: __hostname__ 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enforce_single_del_contracts: true 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_background_jobs: 2 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_background_compactions: -1 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_subcompactions: 1 2023-12-07T17:04:20.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.delayed_write_rate : 16777216 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_total_wal_size: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.stats_dump_period_sec: 600 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.stats_persist_period_sec: 600 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_open_files: -1 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bytes_per_sync: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.wal_bytes_per_sync: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.strict_bytes_per_sync: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_readahead_size: 0 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_background_flushes: -1 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Compression algorithms supported: 2023-12-07T17:04:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kZSTD supported: 0 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kXpressCompression supported: 0 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kLZ4HCCompression supported: 1 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kZlibCompression supported: 1 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kSnappyCompression supported: 1 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kLZ4Compression supported: 1 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: kBZip2Compression supported: 0 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Fast CRC32 supported: Supported on x86 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: DMutex implementation: pthread_mutex_t 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.merge_operator: 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_filter: None 2023-12-07T17:04:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_filter_factory: None 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.sst_partitioner_factory: None 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.memtable_factory: SkipListFactory 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.table_factory: BlockBasedTable 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5574ae8bf8a0) 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cache_index_and_filter_blocks: 1 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: pin_top_level_index_and_filter: 1 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: index_type: 0 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: data_block_index_type: 0 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: index_shortening: 1 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: data_block_hash_table_util_ratio: 0.750000 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: checksum: 4 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: no_block_cache: 0 2023-12-07T17:04:20.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_cache: 0x5574ae913090 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_cache_name: BinnedLRUCache 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_cache_options: 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: capacity : 536870912 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: num_shard_bits : 4 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: strict_capacity_limit : 0 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: high_pri_pool_ratio: 0.000 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_cache_compressed: (nil) 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: persistent_cache: (nil) 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_size: 4096 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_size_deviation: 10 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_restart_interval: 16 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: index_block_restart_interval: 1 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: metadata_block_size: 4096 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: partition_filters: 0 2023-12-07T17:04:20.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: use_delta_encoding: 1 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: filter_policy: bloomfilter 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: whole_key_filtering: 1 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: verify_compression: 0 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: read_amp_bytes_per_bit: 0 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: format_version: 5 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: enable_index_compression: 1 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: block_align: 0 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: max_auto_readahead_size: 262144 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: prepopulate_block_cache: 0 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: initial_auto_readahead_size: 8192 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: num_file_reads_for_auto_readahead: 2 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.write_buffer_size: 33554432 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_write_buffer_number: 2 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression: NoCompression 2023-12-07T17:04:20.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression: Disabled 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.prefix_extractor: nullptr 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.num_levels: 7 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-07T17:04:20.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.window_bits: -14 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.level: 32767 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.strategy: 0 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.enabled: false 2023-12-07T17:04:20.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-07T17:04:20.627 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-07T17:04:20.627 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.target_file_size_base: 67108864 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.target_file_size_multiplier: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-07T17:04:20.628 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.arena_block_size: 1048576 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.disable_auto_compactions: 0 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-07T17:04:20.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.table_properties_collectors: 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.inplace_update_support: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.inplace_update_num_locks: 10000 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.memtable_huge_page_size: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.bloom_locality: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.max_successive_merges: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.optimize_filters_for_hits: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.paranoid_file_checks: 0 2023-12-07T17:04:20.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.force_consistency_checks: 1 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.report_bg_io_stats: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.ttl: 2592000 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.periodic_compaction_seconds: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enable_blob_files: false 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.min_blob_size: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_file_size: 268435456 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_compression_type: NoCompression 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.enable_blob_garbage_collection: false 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.blob_file_starting_level: 0 2023-12-07T17:04:20.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 31bb0bed-33a5-44c1-be3c-a3f6d48e82a3 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701968660470256, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701968660471332, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80519, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78706, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 73014, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701968660, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "31bb0bed-33a5-44c1-be3c-a3f6d48e82a3", "db_session_id": "VZG8BM7R4X3ZHGEVZ3TV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701968660471442, "job": 1, "event": "recovery_finished"} 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.467+0000 7f1b1817ec80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b1817ec80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b1817ec80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5574ae9b2000 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b1817ec80 4 rocksdb: DB pointer 0x5574ae9a0000 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b06936700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b06936700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-07T17:04:20.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: ** DB Stats ** 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: ** Compaction Stats [default] ** 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: L0 2/0 80.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Sum 2/0 80.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: ** Compaction Stats [default] ** 2023-12-07T17:04:20.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 116.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Flush(GB): cumulative 0.000, interval 0.000 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: AddFile(Total Files): cumulative 0, interval 0 2023-12-07T17:04:20.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: AddFile(L0 Files): cumulative 0, interval 0 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: AddFile(Keys): cumulative 0, interval 0 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Cumulative compaction: 0.00 GB write, 16.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Interval compaction: 0.00 GB write, 16.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Block cache BinnedLRUCache@0x5574ae913090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: ** File Read Latency Histogram By Level [default] ** 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b1817ec80 0 starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.471+0000 7f1b1817ec80 1 mon.a@-1(???) e1 preinit fsid 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).mds e1 new map 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).mds e1 print_map 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: e1 2023-12-07T17:04:20.635 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: enable_multiple, ever_enabled_multiple: 1,1 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: legacy client fscid: -1 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: No filesystems configured 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: debug 2023-12-07T17:04:20.475+0000 7f1b1817ec80 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cluster 2023-12-07T17:04:20.479586+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cluster 2023-12-07T17:04:20.479636+0000 mon.a (mon.0) 2 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cluster 2023-12-07T17:04:20.480116+0000 mon.a (mon.0) 3 : cluster 0 fsmap 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cluster 2023-12-07T17:04:20.480161+0000 mon.a (mon.0) 4 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2023-12-07T17:04:20.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:20 smithi064 bash[25056]: cluster 2023-12-07T17:04:20.480565+0000 mon.a (mon.0) 5 : cluster 0 mgrmap e1: no daemons active 2023-12-07T17:04:21.453 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-07T17:04:21.477 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:04:21.477 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2023-12-07T17:04:21.477 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-07T17:04:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-07T17:04:21.777 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y 2023-12-07T17:04:21.778 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y.service: Unit ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y.service not loaded. 2023-12-07T17:04:21.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:21 smithi064 bash[25056]: audit 2023-12-07T17:04:20.770605+0000 mon.a (mon.0) 6 : audit 1 from='client.? 172.21.15.64:0/108887759' entity='client.admin' 2023-12-07T17:04:21.994 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715.target.wants/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@.service. 2023-12-07T17:04:22.006 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld does not appear to be present 2023-12-07T17:04:22.006 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2023-12-07T17:04:22.006 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld does not appear to be present 2023-12-07T17:04:22.006 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-07T17:04:22.006 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2023-12-07T17:04:22.007 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2023-12-07T17:04:23.998 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:23 smithi064 bash[25056]: audit 2023-12-07T17:04:23.788574+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.64:0/82321440' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T17:04:23.998 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:23 smithi064 bash[25333]: debug 2023-12-07T17:04:23.659+0000 7ff3b4025200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:04:24.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:24.397 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "7ae18f76-9522-11ee-95a2-87774f69a715", 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T17:04:24.399 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T17:04:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T17:04:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T17:04:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T17:04:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T17:04:24.400 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T17:04:24.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T17:04:24.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T17:04:24.402 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:24.403 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T17:04:13.886576+0000", 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:24.404 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2023-12-07T17:04:25.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:24 smithi064 bash[25333]: debug 2023-12-07T17:04:24.655+0000 7ff3b4025200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:04:25.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:24 smithi064 bash[25333]: debug 2023-12-07T17:04:24.759+0000 7ff3b4025200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:04:25.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:24 smithi064 bash[25333]: debug 2023-12-07T17:04:24.867+0000 7ff3b4025200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:04:25.510 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:25 smithi064 bash[25333]: debug 2023-12-07T17:04:25.139+0000 7ff3b4025200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:04:25.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:25 smithi064 bash[25333]: debug 2023-12-07T17:04:25.515+0000 7ff3b4025200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:04:25.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:25 smithi064 bash[25333]: debug 2023-12-07T17:04:25.675+0000 7ff3b4025200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:04:26.362 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:25 smithi064 bash[25333]: debug 2023-12-07T17:04:25.899+0000 7ff3b4025200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:04:27.274 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:26 smithi064 bash[25333]: debug 2023-12-07T17:04:26.955+0000 7ff3b4025200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:04:27.275 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:27 smithi064 bash[25333]: debug 2023-12-07T17:04:27.075+0000 7ff3b4025200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:04:27.275 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:27 smithi064 bash[25333]: debug 2023-12-07T17:04:27.195+0000 7ff3b4025200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:04:27.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:27 smithi064 bash[25056]: audit 2023-12-07T17:04:27.628387+0000 mon.a (mon.0) 8 : audit 0 from='client.? 172.21.15.64:0/1382285358' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T17:04:27.939 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:27 smithi064 bash[25333]: debug 2023-12-07T17:04:27.711+0000 7ff3b4025200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:04:28.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:27 smithi064 bash[25333]: debug 2023-12-07T17:04:27.943+0000 7ff3b4025200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "7ae18f76-9522-11ee-95a2-87774f69a715", 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T17:04:28.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T17:04:28.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T17:04:28.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T17:04:28.233 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T17:04:28.234 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T17:04:13.886576+0000", 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T17:04:28.236 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:28.236 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2023-12-07T17:04:28.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:28 smithi064 bash[25333]: debug 2023-12-07T17:04:28.411+0000 7ff3b4025200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:04:29.362 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:28 smithi064 bash[25333]: debug 2023-12-07T17:04:28.927+0000 7ff3b4025200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:04:29.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:29 smithi064 bash[25333]: debug 2023-12-07T17:04:29.511+0000 7ff3b4025200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:04:29.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:29 smithi064 bash[25333]: debug 2023-12-07T17:04:29.623+0000 7ff3b4025200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:04:30.241 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:29 smithi064 bash[25333]: debug 2023-12-07T17:04:29.979+0000 7ff3b4025200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:04:30.241 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.095+0000 7ff3b4025200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:04:30.495 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.243+0000 7ff3b4025200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:04:30.495 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.363+0000 7ff3b4025200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:04:30.612 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.499+0000 7ff3b4025200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:04:30.809 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.691+0000 7ff3b4025200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:04:31.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:30 smithi064 bash[25333]: debug 2023-12-07T17:04:30.811+0000 7ff3b4025200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:04:31.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:31 smithi064 bash[25056]: audit 2023-12-07T17:04:31.379646+0000 mon.a (mon.0) 9 : audit 0 from='client.? 172.21.15.64:0/3066767581' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "7ae18f76-9522-11ee-95a2-87774f69a715", 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T17:04:31.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T17:04:31.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T17:04:31.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:31.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T17:04:31.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T17:04:31.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T17:04:31.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T17:04:31.955 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T17:04:31.956 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T17:04:31.956 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T17:04:31.956 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T17:04:31.956 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T17:04:31.956 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:31.957 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T17:04:13.886576+0000", 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (3/15)... 2023-12-07T17:04:32.362 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25333]: debug 2023-12-07T17:04:32.083+0000 7ff3b4025200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: cluster 2023-12-07T17:04:32.106679+0000 mon.a (mon.0) 10 : cluster 1 Activating manager daemon y 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: cluster 2023-12-07T17:04:32.113187+0000 mon.a (mon.0) 11 : cluster 0 mgrmap e2: y(active, starting, since 0.00680181s) 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.114760+0000 mon.a (mon.0) 12 : audit 0 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.114951+0000 mon.a (mon.0) 13 : audit 0 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.115142+0000 mon.a (mon.0) 14 : audit 0 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.115315+0000 mon.a (mon.0) 15 : audit 0 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.116925+0000 mon.a (mon.0) 16 : audit 0 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: cluster 2023-12-07T17:04:32.144212+0000 mon.a (mon.0) 17 : cluster 1 Manager daemon y is now available 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.165570+0000 mon.a (mon.0) 18 : audit 1 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.169874+0000 mon.a (mon.0) 19 : audit 1 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:04:32.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.175434+0000 mon.a (mon.0) 20 : audit 1 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' 2023-12-07T17:04:32.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.178868+0000 mon.a (mon.0) 21 : audit 1 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' 2023-12-07T17:04:32.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:32 smithi064 bash[25056]: audit 2023-12-07T17:04:32.181249+0000 mon.a (mon.0) 22 : audit 1 from='mgr.14100 172.21.15.64:0/511084898' entity='mgr.y' 2023-12-07T17:04:34.363 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:34 smithi064 bash[25056]: cluster 2023-12-07T17:04:33.120597+0000 mon.a (mon.0) 23 : cluster 0 mgrmap e3: y(active, since 1.01424s) 2023-12-07T17:04:36.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:36.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:36.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "7ae18f76-9522-11ee-95a2-87774f69a715", 2023-12-07T17:04:36.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T17:04:36.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T17:04:36.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:36.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:36.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T17:04:36.230 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T17:04:36.231 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T17:04:13.886576+0000", 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:36.232 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2023-12-07T17:04:36.363 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:36 smithi064 bash[25056]: cluster 2023-12-07T17:04:35.119527+0000 mon.a (mon.0) 24 : cluster 0 mgrmap e4: y(active, since 3s) 2023-12-07T17:04:36.363 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:36 smithi064 bash[25056]: audit 2023-12-07T17:04:35.614602+0000 mon.a (mon.0) 25 : audit 0 from='client.? 172.21.15.64:0/4165299770' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T17:04:38.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:37 smithi064 bash[25056]: audit 2023-12-07T17:04:37.631802+0000 mon.a (mon.0) 26 : audit 1 from='client.? 172.21.15.64:0/1450228207' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-07T17:04:38.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:37 smithi064 bash[25056]: audit 2023-12-07T17:04:37.636671+0000 mon.a (mon.0) 27 : audit 1 from='client.? 172.21.15.64:0/1450228207' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-07T17:04:38.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:38.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2023-12-07T17:04:38.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:38.194 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-07T17:04:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-07T17:04:38.197 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2023-12-07T17:04:40.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:39 smithi064 bash[25056]: audit 2023-12-07T17:04:39.641440+0000 mon.a (mon.0) 28 : audit 1 from='client.? 172.21.15.64:0/3289802585' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-07T17:04:40.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:39 smithi064 bash[25333]: ignoring --setuser ceph since I am not root 2023-12-07T17:04:40.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:39 smithi064 bash[25333]: ignoring --setgroup ceph since I am not root 2023-12-07T17:04:40.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:39 smithi064 bash[25333]: debug 2023-12-07T17:04:39.902+0000 7f166b107200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:04:40.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:40 smithi064 bash[25056]: audit 2023-12-07T17:04:39.693172+0000 mon.a (mon.0) 29 : audit 1 from='client.? 172.21.15.64:0/3289802585' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-07T17:04:40.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:40 smithi064 bash[25056]: cluster 2023-12-07T17:04:39.695952+0000 mon.a (mon.0) 30 : cluster 0 mgrmap e5: y(active, since 7s) 2023-12-07T17:04:40.994 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:40 smithi064 bash[25333]: debug 2023-12-07T17:04:40.830+0000 7f166b107200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:04:40.994 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:40 smithi064 bash[25333]: debug 2023-12-07T17:04:40.938+0000 7f166b107200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:04:41.328 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:41 smithi064 bash[25333]: debug 2023-12-07T17:04:41.046+0000 7f166b107200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:04:41.612 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:41 smithi064 bash[25333]: debug 2023-12-07T17:04:41.330+0000 7f166b107200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:04:42.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:41 smithi064 bash[25056]: audit 2023-12-07T17:04:41.693226+0000 mon.a (mon.0) 31 : audit 0 from='client.? 172.21.15.64:0/3595819638' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-07T17:04:42.092 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:41 smithi064 bash[25333]: debug 2023-12-07T17:04:41.702+0000 7f166b107200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:04:42.092 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:41 smithi064 bash[25333]: debug 2023-12-07T17:04:41.866+0000 7f166b107200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "y", 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-07T17:04:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:42.275 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2023-12-07T17:04:42.275 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2023-12-07T17:04:42.299 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:42 smithi064 bash[25333]: debug 2023-12-07T17:04:42.094+0000 7f166b107200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:04:43.248 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:43 smithi064 bash[25333]: debug 2023-12-07T17:04:43.130+0000 7f166b107200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:04:43.249 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:43 smithi064 bash[25333]: debug 2023-12-07T17:04:43.250+0000 7f166b107200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:04:43.612 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:43 smithi064 bash[25333]: debug 2023-12-07T17:04:43.370+0000 7f166b107200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:04:44.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:43 smithi064 bash[25333]: debug 2023-12-07T17:04:43.858+0000 7f166b107200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:04:44.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:44 smithi064 bash[25333]: debug 2023-12-07T17:04:44.090+0000 7f166b107200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:04:44.862 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:44 smithi064 bash[25333]: debug 2023-12-07T17:04:44.558+0000 7f166b107200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:04:45.362 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:45 smithi064 bash[25333]: debug 2023-12-07T17:04:45.062+0000 7f166b107200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:04:46.102 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:45 smithi064 bash[25333]: debug 2023-12-07T17:04:45.638+0000 7f166b107200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:04:46.102 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:45 smithi064 bash[25333]: debug 2023-12-07T17:04:45.754+0000 7f166b107200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:04:46.362 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.102+0000 7f166b107200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:04:46.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.222+0000 7f166b107200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:04:46.800 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.366+0000 7f166b107200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:04:46.800 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.486+0000 7f166b107200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:04:46.800 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.610+0000 7f166b107200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:04:47.112 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.802+0000 7f166b107200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:04:47.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:46 smithi064 bash[25333]: debug 2023-12-07T17:04:46.930+0000 7f166b107200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25333]: debug 2023-12-07T17:04:48.198+0000 7f166b107200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: cluster 2023-12-07T17:04:48.222145+0000 mon.a (mon.0) 32 : cluster 1 Active manager daemon y restarted 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: cluster 2023-12-07T17:04:48.222917+0000 mon.a (mon.0) 33 : cluster 1 Activating manager daemon y 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: cluster 2023-12-07T17:04:48.229233+0000 mon.a (mon.0) 34 : cluster 0 osdmap e2: 0 total, 0 up, 0 in 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: cluster 2023-12-07T17:04:48.229342+0000 mon.a (mon.0) 35 : cluster 0 mgrmap e6: y(active, starting, since 0.00676721s) 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: audit 2023-12-07T17:04:48.232363+0000 mon.a (mon.0) 36 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:04:48.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: audit 2023-12-07T17:04:48.233744+0000 mon.a (mon.0) 37 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:04:48.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: audit 2023-12-07T17:04:48.234416+0000 mon.a (mon.0) 38 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:04:48.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: audit 2023-12-07T17:04:48.234628+0000 mon.a (mon.0) 39 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:04:48.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: audit 2023-12-07T17:04:48.234831+0000 mon.a (mon.0) 40 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:04:48.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:48 smithi064 bash[25056]: cluster 2023-12-07T17:04:48.263704+0000 mon.a (mon.0) 41 : cluster 1 Manager daemon y is now available 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: cephadm 2023-12-07T17:04:48.290307+0000 mgr.y (mgr.14120) 1 : cephadm 1 Found migration_current of "None". Setting to last migration. 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.295712+0000 mon.a (mon.0) 42 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.299912+0000 mon.a (mon.0) 43 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.305147+0000 mon.a (mon.0) 44 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.308483+0000 mon.a (mon.0) 45 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:04:49.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.313029+0000 mon.a (mon.0) 46 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:04:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.329215+0000 mon.a (mon.0) 47 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:04:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.333642+0000 mon.a (mon.0) 48 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:04:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.804956+0000 mon.a (mon.0) 49 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: audit 2023-12-07T17:04:48.810195+0000 mon.a (mon.0) 50 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:49 smithi064 bash[25056]: cluster 2023-12-07T17:04:49.230386+0000 mon.a (mon.0) 51 : cluster 0 mgrmap e7: y(active, since 1.00781s) 2023-12-07T17:04:49.875 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:04:49.875 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-07T17:04:49.875 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-07T17:04:49.875 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:04:49.876 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2023-12-07T17:04:49.876 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: audit 2023-12-07T17:04:49.231752+0000 mgr.y (mgr.14120) 2 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: audit 2023-12-07T17:04:49.247158+0000 mgr.y (mgr.14120) 3 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: cephadm 2023-12-07T17:04:49.322506+0000 mgr.y (mgr.14120) 4 : cephadm 1 [07/Dec/2023:17:04:49] ENGINE Bus STARTING 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: cephadm 2023-12-07T17:04:49.424608+0000 mgr.y (mgr.14120) 5 : cephadm 1 [07/Dec/2023:17:04:49] ENGINE Serving on http://172.21.15.64:8765 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: cephadm 2023-12-07T17:04:49.556034+0000 mgr.y (mgr.14120) 6 : cephadm 1 [07/Dec/2023:17:04:49] ENGINE Serving on https://172.21.15.64:7150 2023-12-07T17:04:50.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: cephadm 2023-12-07T17:04:49.556388+0000 mgr.y (mgr.14120) 7 : cephadm 1 [07/Dec/2023:17:04:49] ENGINE Bus STARTED 2023-12-07T17:04:50.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:50 smithi064 bash[25056]: audit 2023-12-07T17:04:49.589304+0000 mon.a (mon.0) 52 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:04:51.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:51 smithi064 bash[25056]: cluster 2023-12-07T17:04:50.299855+0000 mon.a (mon.0) 53 : cluster 0 mgrmap e8: y(active, since 2s) 2023-12-07T17:04:51.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:51 smithi064 bash[25056]: audit 2023-12-07T17:04:51.244214+0000 mon.a (mon.0) 54 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:52.567 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:52 smithi064 bash[25056]: audit 2023-12-07T17:04:51.235733+0000 mgr.y (mgr.14120) 8 : audit 0 from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:04:52.567 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:52 smithi064 bash[25056]: audit 2023-12-07T17:04:51.311858+0000 mon.a (mon.0) 55 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:04:53.786 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-07T17:04:53.787 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2023-12-07T17:04:54.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:53 smithi064 bash[25056]: audit 2023-12-07T17:04:53.163700+0000 mgr.y (mgr.14120) 9 : audit 0 from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: Generating public/private rsa key pair. 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: Your identification has been saved in /tmp/tmpg9kkid_h/key. 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: Your public key has been saved in /tmp/tmpg9kkid_h/key.pub. 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: The key fingerprint is: 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: SHA256:9vka/19fIUHLjUGVxvNvIhMjGXKFBv65VWY9K7vT0aw ceph-7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: The key's randomart image is: 2023-12-07T17:04:55.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: +---[RSA 3072]----+ 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | .. oo+o..| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | .. = o =* | 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | .+ o ==o+| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | .o.o+. +| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | S o..= +o| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | . . +o =.*| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | = +.++| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | + .E.+| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: | ..ooo.o| 2023-12-07T17:04:55.614 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:04:55 smithi064 bash[25333]: +----[SHA256]-----+ 2023-12-07T17:04:56.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:56 smithi064 bash[25056]: audit 2023-12-07T17:04:55.186395+0000 mgr.y (mgr.14120) 10 : audit 0 from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:04:56.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:56 smithi064 bash[25056]: cephadm 2023-12-07T17:04:55.186665+0000 mgr.y (mgr.14120) 11 : cephadm 1 Generating ssh key... 2023-12-07T17:04:56.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:56 smithi064 bash[25056]: audit 2023-12-07T17:04:55.315150+0000 mon.a (mon.0) 56 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:56.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:56 smithi064 bash[25056]: audit 2023-12-07T17:04:55.318975+0000 mon.a (mon.0) 57 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:04:57.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41O9eLnBt/RXoJpfkerX0tdcHaYvuGsWgUNbbamg6ndoxbP0+j5F6/p6G1n2h7zDfOGsIQ4lY0nWuCO5kgCp8D6xvU6emnbr+lvGqNX4z7tWcsqCTgTy1Wbexl4TlZDxLMqkvuE8zFcL/NAdeFPQW6gWNz9JniasuSxqHVNzbCjB+f/gl8V0X7+wYptHNOOybSWBbDLHjbouBEiSEqP0/+eMP+16763qV3XlSgKYBgPa9DB0e0tsO73m01dzmEN61uqCtuRf6jGymb3oZvDT3p1KSfQ8gF+fCTKVDfDCCrClYvR8Woa37YKk9xyFLe+uCZ4wOsqQu2XVwKYchhCkIYYSFs4vg++rvx0XVzunhqOol5Odzr4MzY08mkvhDzAIzvf1ITD/XjQHB1fjZ/rzjhh3G9S8Ndb/l0tQ3UyF7PbrYBDhGvf4OVZbXDsPZUBbA4RrLaPDHncnI6rUAtx/r6R2l7KVRmgx4x3gHYAOPbvo166HzP4vYJnH6oAWmi0U= ceph-7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:04:57.886 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-07T17:04:57.886 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2023-12-07T17:04:57.886 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2023-12-07T17:04:58.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:57 smithi064 bash[25056]: audit 2023-12-07T17:04:57.284420+0000 mgr.y (mgr.14120) 12 : audit 0 from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:04:59.813 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:04:59 smithi064 bash[25056]: audit 2023-12-07T17:04:59.205895+0000 mgr.y (mgr.14120) 13 : audit 0 from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:01.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:01 smithi064 bash[25056]: cephadm 2023-12-07T17:05:00.017401+0000 mgr.y (mgr.14120) 14 : cephadm 1 Deploying cephadm binary to smithi064 2023-12-07T17:05:06.585 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2023-12-07T17:05:06.585 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2023-12-07T17:05:06.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:06 smithi064 bash[25056]: audit 2023-12-07T17:05:05.535231+0000 mon.a (mon.0) 58 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:05:06.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:06 smithi064 bash[25056]: cephadm 2023-12-07T17:05:05.536107+0000 mgr.y (mgr.14120) 15 : cephadm 1 Added host smithi064 2023-12-07T17:05:06.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:06 smithi064 bash[25056]: audit 2023-12-07T17:05:05.580794+0000 mon.a (mon.0) 59 : audit 0 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:09.129 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-07T17:05:09.129 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2023-12-07T17:05:09.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: cluster 2023-12-07T17:05:08.236081+0000 mgr.y (mgr.14120) 16 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:09.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: audit 2023-12-07T17:05:08.404840+0000 mgr.y (mgr.14120) 17 : audit 0 from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:09.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: cephadm 2023-12-07T17:05:08.408128+0000 mgr.y (mgr.14120) 18 : cephadm 1 Saving service mon spec with placement count:5 2023-12-07T17:05:09.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: audit 2023-12-07T17:05:08.415236+0000 mon.a (mon.0) 60 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:05:09.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: audit 2023-12-07T17:05:08.807919+0000 mon.a (mon.0) 61 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:05:09.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:09 smithi064 bash[25056]: audit 2023-12-07T17:05:09.342350+0000 mon.a (mon.0) 62 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:05:11.202 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-07T17:05:11.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:11 smithi064 bash[25056]: cluster 2023-12-07T17:05:10.236521+0000 mgr.y (mgr.14120) 19 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:11.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:11 smithi064 bash[25056]: audit 2023-12-07T17:05:10.540626+0000 mgr.y (mgr.14120) 20 : audit 0 from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:11.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:11 smithi064 bash[25056]: cephadm 2023-12-07T17:05:10.543276+0000 mgr.y (mgr.14120) 21 : cephadm 1 Saving service mgr spec with placement count:2 2023-12-07T17:05:11.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:11 smithi064 bash[25056]: audit 2023-12-07T17:05:10.547837+0000 mon.a (mon.0) 63 : audit 1 from='mgr.14120 172.21.15.64:0/1302830402' entity='mgr.y' 2023-12-07T17:05:13.821 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:13 smithi064 bash[25056]: cluster 2023-12-07T17:05:12.236893+0000 mgr.y (mgr.14120) 22 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:13.821 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:13 smithi064 bash[25056]: audit 2023-12-07T17:05:12.540509+0000 mon.a (mon.0) 64 : audit 1 from='client.? 172.21.15.64:0/44766980' entity='client.admin' 2023-12-07T17:05:15.216 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2023-12-07T17:05:15.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:15 smithi064 bash[25056]: cluster 2023-12-07T17:05:14.237279+0000 mgr.y (mgr.14120) 23 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:15.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:15 smithi064 bash[25056]: audit 2023-12-07T17:05:14.511393+0000 mon.a (mon.0) 65 : audit 1 from='client.? 172.21.15.64:0/3632495750' entity='client.admin' 2023-12-07T17:05:17.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:17 smithi064 bash[25056]: cluster 2023-12-07T17:05:16.237675+0000 mgr.y (mgr.14120) 24 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:17.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:17 smithi064 bash[25056]: audit 2023-12-07T17:05:16.702608+0000 mon.a (mon.0) 66 : audit 1 from='client.? 172.21.15.64:0/2518310610' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-07T17:05:17.768 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:17 smithi064 bash[25333]: ignoring --setuser ceph since I am not root 2023-12-07T17:05:17.768 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:17 smithi064 bash[25333]: ignoring --setgroup ceph since I am not root 2023-12-07T17:05:17.768 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:17 smithi064 bash[25333]: debug 2023-12-07T17:05:17.745+0000 7f8463810200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:05:18.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:18 smithi064 bash[25056]: audit 2023-12-07T17:05:17.517735+0000 mon.a (mon.0) 67 : audit 1 from='client.? 172.21.15.64:0/2518310610' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-07T17:05:18.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:18 smithi064 bash[25056]: cluster 2023-12-07T17:05:17.522892+0000 mon.a (mon.0) 68 : cluster 0 mgrmap e9: y(active, since 29s) 2023-12-07T17:05:18.838 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:18 smithi064 bash[25333]: debug 2023-12-07T17:05:18.733+0000 7f8463810200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:05:19.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:18 smithi064 bash[25333]: debug 2023-12-07T17:05:18.837+0000 7f8463810200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:05:19.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:18 smithi064 bash[25333]: debug 2023-12-07T17:05:18.941+0000 7f8463810200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:05:19.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:19 smithi064 bash[25333]: debug 2023-12-07T17:05:19.229+0000 7f8463810200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:05:19.958 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:19 smithi064 bash[25333]: debug 2023-12-07T17:05:19.613+0000 7f8463810200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:05:19.958 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:19 smithi064 bash[25333]: debug 2023-12-07T17:05:19.777+0000 7f8463810200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:05:19.958 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:19 smithi064 bash[25056]: audit 2023-12-07T17:05:19.732976+0000 mon.a (mon.0) 69 : audit 0 from='client.? 172.21.15.64:0/2621228309' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-07T17:05:20.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:20 smithi064 bash[25333]: debug 2023-12-07T17:05:20.005+0000 7f8463810200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "y", 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2023-12-07T17:05:20.386 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2023-12-07T17:05:21.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:21 smithi064 bash[25333]: debug 2023-12-07T17:05:21.049+0000 7f8463810200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:05:21.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:21 smithi064 bash[25333]: debug 2023-12-07T17:05:21.165+0000 7f8463810200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:05:21.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:21 smithi064 bash[25333]: debug 2023-12-07T17:05:21.297+0000 7f8463810200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:05:22.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:21 smithi064 bash[25333]: debug 2023-12-07T17:05:21.825+0000 7f8463810200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:05:22.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:22 smithi064 bash[25333]: debug 2023-12-07T17:05:22.057+0000 7f8463810200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:05:22.863 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:22 smithi064 bash[25333]: debug 2023-12-07T17:05:22.533+0000 7f8463810200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:05:23.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:23 smithi064 bash[25333]: debug 2023-12-07T17:05:23.041+0000 7f8463810200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:05:24.092 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:23 smithi064 bash[25333]: debug 2023-12-07T17:05:23.621+0000 7f8463810200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:05:24.093 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:23 smithi064 bash[25333]: debug 2023-12-07T17:05:23.737+0000 7f8463810200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:05:24.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.093+0000 7f8463810200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:05:24.363 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.213+0000 7f8463810200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:05:24.813 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.365+0000 7f8463810200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:05:24.813 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.485+0000 7f8463810200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:05:24.813 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.617+0000 7f8463810200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:05:25.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.813+0000 7f8463810200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:05:25.113 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:24 smithi064 bash[25333]: debug 2023-12-07T17:05:24.945+0000 7f8463810200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:05:26.613 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25333]: debug 2023-12-07T17:05:26.253+0000 7f8463810200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:05:26.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: cluster 2023-12-07T17:05:26.278482+0000 mon.a (mon.0) 70 : cluster 1 Active manager daemon y restarted 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: cluster 2023-12-07T17:05:26.279152+0000 mon.a (mon.0) 71 : cluster 1 Activating manager daemon y 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: cluster 2023-12-07T17:05:26.286120+0000 mon.a (mon.0) 72 : cluster 0 osdmap e3: 0 total, 0 up, 0 in 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: cluster 2023-12-07T17:05:26.286391+0000 mon.a (mon.0) 73 : cluster 0 mgrmap e10: y(active, starting, since 0.00745768s) 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.287585+0000 mon.a (mon.0) 74 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.288786+0000 mon.a (mon.0) 75 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.289655+0000 mon.a (mon.0) 76 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.289885+0000 mon.a (mon.0) 77 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.290041+0000 mon.a (mon.0) 78 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: cluster 2023-12-07T17:05:26.315990+0000 mon.a (mon.0) 79 : cluster 1 Manager daemon y is now available 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.345704+0000 mon.a (mon.0) 80 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.351505+0000 mon.a (mon.0) 81 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.379185+0000 mon.a (mon.0) 82 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:05:26.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:26 smithi064 bash[25056]: audit 2023-12-07T17:05:26.383186+0000 mon.a (mon.0) 83 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2023-12-07T17:05:27.972 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2023-12-07T17:05:28.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: cephadm 2023-12-07T17:05:26.868898+0000 mgr.y (mgr.14152) 1 : cephadm 1 [07/Dec/2023:17:05:26] ENGINE Bus STARTING 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: cephadm 2023-12-07T17:05:26.995362+0000 mgr.y (mgr.14152) 2 : cephadm 1 [07/Dec/2023:17:05:26] ENGINE Serving on https://172.21.15.64:7150 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: cephadm 2023-12-07T17:05:27.097476+0000 mgr.y (mgr.14152) 3 : cephadm 1 [07/Dec/2023:17:05:27] ENGINE Serving on http://172.21.15.64:8765 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: cephadm 2023-12-07T17:05:27.097800+0000 mgr.y (mgr.14152) 4 : cephadm 1 [07/Dec/2023:17:05:27] ENGINE Bus STARTED 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: audit 2023-12-07T17:05:27.290471+0000 mgr.y (mgr.14152) 5 : audit 0 from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: cluster 2023-12-07T17:05:27.292518+0000 mon.a (mon.0) 84 : cluster 0 mgrmap e11: y(active, since 1.01359s) 2023-12-07T17:05:28.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:28 smithi064 bash[25056]: audit 2023-12-07T17:05:27.306152+0000 mgr.y (mgr.14152) 6 : audit 0 from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-07T17:05:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-07T17:05:30.150 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2023-12-07T17:05:30.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:30 smithi064 bash[25056]: cluster 2023-12-07T17:05:29.291070+0000 mon.a (mon.0) 85 : cluster 0 mgrmap e12: y(active, since 3s) 2023-12-07T17:05:30.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:30 smithi064 bash[25056]: audit 2023-12-07T17:05:29.318432+0000 mgr.y (mgr.14152) 7 : audit 0 from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:30.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:30 smithi064 bash[25056]: audit 2023-12-07T17:05:29.480583+0000 mon.a (mon.0) 86 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:30.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:30 smithi064 bash[25056]: audit 2023-12-07T17:05:29.484737+0000 mon.a (mon.0) 87 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:32.048 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:31 smithi064 bash[25056]: audit 2023-12-07T17:05:31.536073+0000 mgr.y (mgr.14152) 8 : audit 0 from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:32.467 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zONsdD34n153zdZ3T.w8OO8qRj7FWJFgpTzAMkuirMfrK7IPEka.6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701968731, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-07T17:05:32.467 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2023-12-07T17:05:33.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:32 smithi064 bash[25056]: audit 2023-12-07T17:05:31.800957+0000 mon.a (mon.0) 88 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:33 smithi064 bash[25056]: cluster 2023-12-07T17:05:32.808967+0000 mon.a (mon.0) 89 : cluster 0 mgrmap e13: y(active, since 6s) 2023-12-07T17:05:34.007 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:33 smithi064 bash[25056]: audit 2023-12-07T17:05:33.259135+0000 mon.a (mon.0) 90 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:34.007 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:33 smithi064 bash[25056]: audit 2023-12-07T17:05:33.767944+0000 mon.a (mon.0) 91 : audit 0 from='client.? 172.21.15.64:0/565656194' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-07T17:05:34.446 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2023-12-07T17:05:34.447 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld does not appear to be present 2023-12-07T17:05:34.447 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-07T17:05:34.450 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout: Password: bhri367rbu 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:34.451 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config directory 2023-12-07T17:05:34.452 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling autotune for osd_memory_target 2023-12-07T17:05:35.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:35 smithi064 bash[25056]: audit 2023-12-07T17:05:34.400215+0000 mon.a (mon.0) 92 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:38.716 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-07T17:05:38.716 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-07T17:05:38.716 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:05:38.717 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2023-12-07T17:05:38.758 INFO:tasks.cephadm:Fetching config... 2023-12-07T17:05:38.758 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:05:38.758 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-07T17:05:38.765 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-07T17:05:38.765 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:05:38.765 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-07T17:05:38.813 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-07T17:05:38.813 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:05:38.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-07T17:05:38.870 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-07T17:05:38.870 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:05:38.870 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-07T17:05:38.917 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-07T17:05:38.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41O9eLnBt/RXoJpfkerX0tdcHaYvuGsWgUNbbamg6ndoxbP0+j5F6/p6G1n2h7zDfOGsIQ4lY0nWuCO5kgCp8D6xvU6emnbr+lvGqNX4z7tWcsqCTgTy1Wbexl4TlZDxLMqkvuE8zFcL/NAdeFPQW6gWNz9JniasuSxqHVNzbCjB+f/gl8V0X7+wYptHNOOybSWBbDLHjbouBEiSEqP0/+eMP+16763qV3XlSgKYBgPa9DB0e0tsO73m01dzmEN61uqCtuRf6jGymb3oZvDT3p1KSfQ8gF+fCTKVDfDCCrClYvR8Woa37YKk9xyFLe+uCZ4wOsqQu2XVwKYchhCkIYYSFs4vg++rvx0XVzunhqOol5Odzr4MzY08mkvhDzAIzvf1ITD/XjQHB1fjZ/rzjhh3G9S8Ndb/l0tQ3UyF7PbrYBDhGvf4OVZbXDsPZUBbA4RrLaPDHncnI6rUAtx/r6R2l7KVRmgx4x3gHYAOPbvo166HzP4vYJnH6oAWmi0U= ceph-7ae18f76-9522-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-07T17:05:38.982 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41O9eLnBt/RXoJpfkerX0tdcHaYvuGsWgUNbbamg6ndoxbP0+j5F6/p6G1n2h7zDfOGsIQ4lY0nWuCO5kgCp8D6xvU6emnbr+lvGqNX4z7tWcsqCTgTy1Wbexl4TlZDxLMqkvuE8zFcL/NAdeFPQW6gWNz9JniasuSxqHVNzbCjB+f/gl8V0X7+wYptHNOOybSWBbDLHjbouBEiSEqP0/+eMP+16763qV3XlSgKYBgPa9DB0e0tsO73m01dzmEN61uqCtuRf6jGymb3oZvDT3p1KSfQ8gF+fCTKVDfDCCrClYvR8Woa37YKk9xyFLe+uCZ4wOsqQu2XVwKYchhCkIYYSFs4vg++rvx0XVzunhqOol5Odzr4MzY08mkvhDzAIzvf1ITD/XjQHB1fjZ/rzjhh3G9S8Ndb/l0tQ3UyF7PbrYBDhGvf4OVZbXDsPZUBbA4RrLaPDHncnI6rUAtx/r6R2l7KVRmgx4x3gHYAOPbvo166HzP4vYJnH6oAWmi0U= ceph-7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:05:38.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41O9eLnBt/RXoJpfkerX0tdcHaYvuGsWgUNbbamg6ndoxbP0+j5F6/p6G1n2h7zDfOGsIQ4lY0nWuCO5kgCp8D6xvU6emnbr+lvGqNX4z7tWcsqCTgTy1Wbexl4TlZDxLMqkvuE8zFcL/NAdeFPQW6gWNz9JniasuSxqHVNzbCjB+f/gl8V0X7+wYptHNOOybSWBbDLHjbouBEiSEqP0/+eMP+16763qV3XlSgKYBgPa9DB0e0tsO73m01dzmEN61uqCtuRf6jGymb3oZvDT3p1KSfQ8gF+fCTKVDfDCCrClYvR8Woa37YKk9xyFLe+uCZ4wOsqQu2XVwKYchhCkIYYSFs4vg++rvx0XVzunhqOol5Odzr4MzY08mkvhDzAIzvf1ITD/XjQHB1fjZ/rzjhh3G9S8Ndb/l0tQ3UyF7PbrYBDhGvf4OVZbXDsPZUBbA4RrLaPDHncnI6rUAtx/r6R2l7KVRmgx4x3gHYAOPbvo166HzP4vYJnH6oAWmi0U= ceph-7ae18f76-9522-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-07T17:05:39.014 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41O9eLnBt/RXoJpfkerX0tdcHaYvuGsWgUNbbamg6ndoxbP0+j5F6/p6G1n2h7zDfOGsIQ4lY0nWuCO5kgCp8D6xvU6emnbr+lvGqNX4z7tWcsqCTgTy1Wbexl4TlZDxLMqkvuE8zFcL/NAdeFPQW6gWNz9JniasuSxqHVNzbCjB+f/gl8V0X7+wYptHNOOybSWBbDLHjbouBEiSEqP0/+eMP+16763qV3XlSgKYBgPa9DB0e0tsO73m01dzmEN61uqCtuRf6jGymb3oZvDT3p1KSfQ8gF+fCTKVDfDCCrClYvR8Woa37YKk9xyFLe+uCZ4wOsqQu2XVwKYchhCkIYYSFs4vg++rvx0XVzunhqOol5Odzr4MzY08mkvhDzAIzvf1ITD/XjQHB1fjZ/rzjhh3G9S8Ndb/l0tQ3UyF7PbrYBDhGvf4OVZbXDsPZUBbA4RrLaPDHncnI6rUAtx/r6R2l7KVRmgx4x3gHYAOPbvo166HzP4vYJnH6oAWmi0U= ceph-7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:05:39.020 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-07T17:05:39.355 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:39 smithi064 bash[25056]: audit 2023-12-07T17:05:38.057924+0000 mon.a (mon.0) 93 : audit 1 from='client.? 172.21.15.64:0/4163823900' entity='client.admin' 2023-12-07T17:05:43.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.748232+0000 mon.a (mon.0) 94 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.753087+0000 mon.a (mon.0) 95 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.754451+0000 mon.a (mon.0) 96 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.761604+0000 mon.a (mon.0) 97 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.796247+0000 mon.a (mon.0) 98 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:45 smithi064 bash[25056]: audit 2023-12-07T17:05:44.805276+0000 mon.a (mon.0) 99 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:47.048 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:46 smithi064 bash[25056]: cluster 2023-12-07T17:05:46.291453+0000 mgr.y (mgr.14152) 9 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:47.487 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-07T17:05:47.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-07T17:05:48.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:47 smithi064 bash[25056]: audit 2023-12-07T17:05:46.825922+0000 mon.a (mon.0) 100 : audit 1 from='client.? 172.21.15.64:0/3904421974' entity='client.admin' 2023-12-07T17:05:48.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:47 smithi064 bash[25056]: audit 2023-12-07T17:05:46.856867+0000 mon.a (mon.0) 101 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:48.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:47 smithi064 bash[25056]: audit 2023-12-07T17:05:46.866202+0000 mon.a (mon.0) 102 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:49.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:48 smithi064 bash[25056]: cluster 2023-12-07T17:05:48.291866+0000 mgr.y (mgr.14152) 10 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:51.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:51 smithi064 bash[25056]: cluster 2023-12-07T17:05:50.292263+0000 mgr.y (mgr.14152) 11 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:52.204 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:05:53.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:53 smithi064 bash[25056]: cluster 2023-12-07T17:05:52.292730+0000 mgr.y (mgr.14152) 12 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:55.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:55 smithi064 bash[25056]: cluster 2023-12-07T17:05:54.293205+0000 mgr.y (mgr.14152) 13 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:55.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:55 smithi064 bash[25056]: audit 2023-12-07T17:05:55.120249+0000 mon.a (mon.0) 103 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:55.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:55 smithi064 bash[25056]: audit 2023-12-07T17:05:55.135263+0000 mon.a (mon.0) 104 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:05:55.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:55 smithi064 bash[25056]: audit 2023-12-07T17:05:55.137986+0000 mon.a (mon.0) 105 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:05:55.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:55 smithi064 bash[25056]: audit 2023-12-07T17:05:55.139514+0000 mon.a (mon.0) 106 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:05:55.778 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2023-12-07T17:05:55.778 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:05:55.778 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2023-12-07T17:05:55.786 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:05:55.786 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:05:55.837 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2023-12-07T17:05:55.837 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch host add smithi184 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: audit 2023-12-07T17:05:55.115251+0000 mgr.y (mgr.14152) 14 : audit 0 from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: cephadm 2023-12-07T17:05:55.141847+0000 mgr.y (mgr.14152) 15 : cephadm 1 Updating smithi064:/etc/ceph/ceph.conf 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: cephadm 2023-12-07T17:05:55.306461+0000 mgr.y (mgr.14152) 16 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: audit 2023-12-07T17:05:55.957765+0000 mon.a (mon.0) 107 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: audit 2023-12-07T17:05:55.963271+0000 mon.a (mon.0) 108 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:56.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:56 smithi064 bash[25056]: audit 2023-12-07T17:05:55.966586+0000 mon.a (mon.0) 109 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:05:57.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:57 smithi064 bash[25056]: cephadm 2023-12-07T17:05:55.621886+0000 mgr.y (mgr.14152) 17 : cephadm 1 Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:05:57.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:57 smithi064 bash[25056]: cephadm 2023-12-07T17:05:55.786776+0000 mgr.y (mgr.14152) 18 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:05:57.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:57 smithi064 bash[25056]: cluster 2023-12-07T17:05:56.293640+0000 mgr.y (mgr.14152) 19 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:05:59.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:05:59 smithi064 bash[25056]: cluster 2023-12-07T17:05:58.294044+0000 mgr.y (mgr.14152) 20 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:06:01.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:01 smithi064 bash[25056]: cluster 2023-12-07T17:06:00.294469+0000 mgr.y (mgr.14152) 21 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:03.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:03 smithi064 bash[25056]: cluster 2023-12-07T17:06:02.294831+0000 mgr.y (mgr.14152) 22 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:04.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:04 smithi064 bash[25056]: audit 2023-12-07T17:06:03.291873+0000 mgr.y (mgr.14152) 23 : audit 0 from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:06:05.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:05 smithi064 bash[25056]: cephadm 2023-12-07T17:06:04.094852+0000 mgr.y (mgr.14152) 24 : cephadm 1 Deploying cephadm binary to smithi184 2023-12-07T17:06:05.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:05 smithi064 bash[25056]: cluster 2023-12-07T17:06:04.295250+0000 mgr.y (mgr.14152) 25 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:07.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:07 smithi064 bash[25056]: cluster 2023-12-07T17:06:06.295600+0000 mgr.y (mgr.14152) 26 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:09.653 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi184' with addr '172.21.15.184' 2023-12-07T17:06:09.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:09 smithi064 bash[25056]: cluster 2023-12-07T17:06:08.295933+0000 mgr.y (mgr.14152) 27 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:10.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-07T17:06:11.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:10 smithi064 bash[25056]: audit 2023-12-07T17:06:09.655670+0000 mon.a (mon.0) 110 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:11.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:10 smithi064 bash[25056]: cephadm 2023-12-07T17:06:09.656575+0000 mgr.y (mgr.14152) 28 : cephadm 1 Added host smithi184 2023-12-07T17:06:11.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:10 smithi064 bash[25056]: audit 2023-12-07T17:06:09.706076+0000 mon.a (mon.0) 111 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:06:11.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:10 smithi064 bash[25056]: audit 2023-12-07T17:06:10.230564+0000 mon.a (mon.0) 112 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:11.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:10 smithi064 bash[25056]: cluster 2023-12-07T17:06:10.296280+0000 mgr.y (mgr.14152) 29 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:13.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:12 smithi064 bash[25056]: audit 2023-12-07T17:06:11.706985+0000 mon.a (mon.0) 113 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:13.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:12 smithi064 bash[25056]: cluster 2023-12-07T17:06:12.296673+0000 mgr.y (mgr.14152) 30 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:14.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:13 smithi064 bash[25056]: audit 2023-12-07T17:06:12.752201+0000 mon.a (mon.0) 114 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:15.113 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:06:15.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:14 smithi064 bash[25056]: cluster 2023-12-07T17:06:14.297109+0000 mgr.y (mgr.14152) 31 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:17.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:17 smithi064 bash[25056]: cluster 2023-12-07T17:06:16.297540+0000 mgr.y (mgr.14152) 32 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:17.992 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:06:17.993 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2023-12-07T17:06:18.667 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-07T17:06:18.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:17.994994+0000 mgr.y (mgr.14152) 33 : audit 0 from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: cluster 2023-12-07T17:06:18.297913+0000 mgr.y (mgr.14152) 34 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:19.492510+0000 mon.a (mon.0) 115 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:19.496391+0000 mon.a (mon.0) 116 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:19.501441+0000 mon.a (mon.0) 117 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:19.504928+0000 mon.a (mon.0) 118 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:19.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:19 smithi064 bash[25056]: audit 2023-12-07T17:06:19.506056+0000 mon.a (mon.0) 119 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: audit 2023-12-07T17:06:19.507768+0000 mon.a (mon.0) 120 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: audit 2023-12-07T17:06:19.514614+0000 mon.a (mon.0) 121 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: cephadm 2023-12-07T17:06:19.516608+0000 mgr.y (mgr.14152) 35 : cephadm 1 Updating smithi184:/etc/ceph/ceph.conf 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: audit 2023-12-07T17:06:20.184185+0000 mon.a (mon.0) 122 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: audit 2023-12-07T17:06:20.188899+0000 mon.a (mon.0) 123 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:20 smithi064 bash[25056]: audit 2023-12-07T17:06:20.193258+0000 mon.a (mon.0) 124 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:21.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:21 smithi064 bash[25056]: cephadm 2023-12-07T17:06:19.681006+0000 mgr.y (mgr.14152) 36 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:06:21.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:21 smithi064 bash[25056]: cephadm 2023-12-07T17:06:19.846027+0000 mgr.y (mgr.14152) 37 : cephadm 1 Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:06:21.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:21 smithi064 bash[25056]: cephadm 2023-12-07T17:06:20.011518+0000 mgr.y (mgr.14152) 38 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:06:21.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:21 smithi064 bash[25056]: cluster 2023-12-07T17:06:20.298328+0000 mgr.y (mgr.14152) 39 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:06:23.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:23 smithi064 bash[25056]: cluster 2023-12-07T17:06:22.298987+0000 mgr.y (mgr.14152) 40 : cluster 0 pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:25.812 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:25 smithi064 bash[25056]: cluster 2023-12-07T17:06:24.299374+0000 mgr.y (mgr.14152) 41 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-12-07T17:06:26.783 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:26 smithi064 bash[25056]: audit 2023-12-07T17:06:26.294834+0000 mon.a (mon.0) 125 : audit 1 from='client.? 172.21.15.64:0/81611152' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-07T17:06:27.224 INFO:tasks.cephadm:Adding mon.a on smithi064 2023-12-07T17:06:27.224 INFO:tasks.cephadm:Adding mon.c on smithi064 2023-12-07T17:06:27.225 INFO:tasks.cephadm:Adding mon.b on smithi184 2023-12-07T17:06:27.225 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi064:[v2:172.21.15.64:3301,v1:172.21.15.64:6790]=c;smithi184:172.21.15.184=b' 2023-12-07T17:06:27.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:27 smithi064 bash[25056]: cluster 2023-12-07T17:06:26.299811+0000 mgr.y (mgr.14152) 42 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:27.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:27 smithi064 bash[25056]: audit 2023-12-07T17:06:26.523966+0000 mon.a (mon.0) 126 : audit 1 from='client.? 172.21.15.64:0/81611152' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-07T17:06:27.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:27 smithi064 bash[25056]: cluster 2023-12-07T17:06:26.528473+0000 mon.a (mon.0) 127 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:28.457 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:06:29.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:29 smithi064 bash[25056]: cluster 2023-12-07T17:06:28.300193+0000 mgr.y (mgr.14152) 43 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:31.228 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2023-12-07T17:06:31.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: cluster 2023-12-07T17:06:30.300652+0000 mgr.y (mgr.14152) 44 : cluster 0 pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:31.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.230098+0000 mon.a (mon.0) 128 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:31.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.253268+0000 mon.a (mon.0) 129 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:06:31.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.257201+0000 mon.a (mon.0) 130 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:31.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.259113+0000 mon.a (mon.0) 131 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:31.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.268300+0000 mon.a (mon.0) 132 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:31.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.272113+0000 mon.a (mon.0) 133 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:31.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:31 smithi064 bash[25056]: audit 2023-12-07T17:06:31.273544+0000 mon.a (mon.0) 134 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:32.058 DEBUG:teuthology.orchestra.run.smithi064:mon.c> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.c.service 2023-12-07T17:06:32.061 DEBUG:teuthology.orchestra.run.smithi184:mon.b> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.b.service 2023-12-07T17:06:32.063 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-07T17:06:32.064 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-07T17:06:32.345 INFO:journalctl@ceph.mon.c.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:06:32.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:06:32.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:32 smithi064 bash[25056]: audit 2023-12-07T17:06:31.219961+0000 mgr.y (mgr.14152) 45 : audit 0 from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi064:[v2:172.21.15.64:3301,v1:172.21.15.64:6790]=c;smithi184:172.21.15.184=b", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:06:32.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:32 smithi064 bash[25056]: cephadm 2023-12-07T17:06:31.225992+0000 mgr.y (mgr.14152) 46 : cephadm 1 Saving service mon spec with placement smithi064:172.21.15.64=a;smithi064:[v2:172.21.15.64:3301,v1:172.21.15.64:6790]=c;smithi184:172.21.15.184=b;count:3 2023-12-07T17:06:32.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:32 smithi064 bash[25056]: cephadm 2023-12-07T17:06:31.275128+0000 mgr.y (mgr.14152) 47 : cephadm 1 Deploying daemon mon.b on smithi184 2023-12-07T17:06:33.327 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:06:33.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:33 smithi064 bash[25056]: cluster 2023-12-07T17:06:32.300966+0000 mgr.y (mgr.14152) 48 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:35.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:35 smithi064 bash[25056]: cluster 2023-12-07T17:06:34.301380+0000 mgr.y (mgr.14152) 49 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:36.783 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-12-07T17:06:36.783 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"7ae18f76-9522-11ee-95a2-87774f69a715","modified":"2023-12-07T17:04:09.272348Z","created":"2023-12-07T17:04:09.272348Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-07T17:06:36.788 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2023-12-07T17:06:37.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: cluster 2023-12-07T17:06:36.301923+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:37.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:36.786802+0000 mon.a (mon.0) 135 : audit 0 from='client.? 172.21.15.184:0/1016004227' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T17:06:37.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:37.085962+0000 mon.a (mon.0) 136 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:37.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:37.091314+0000 mon.a (mon.0) 137 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:37.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:37.108970+0000 mon.a (mon.0) 138 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:37.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:37.110273+0000 mon.a (mon.0) 139 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:37.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:37 smithi064 bash[25056]: audit 2023-12-07T17:06:37.111770+0000 mon.a (mon.0) 140 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:38.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:38 smithi184 bash[31336]: debug 2023-12-07T17:06:38.158+0000 7f4927d26700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-07T17:06:38.775 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-07T17:06:38.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-07T17:06:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:42 smithi064 bash[36207]: debug 2023-12-07T17:06:42.192+0000 7fabceb2d700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-07T17:06:42.494 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:38.168389+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:38.171396+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:38.171723+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:38.196879+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:38.302363+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:39.167691+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:40.168153+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:40.169519+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:40.302778+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:41.168088+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:42.168104+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:42.204613+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:42.303174+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:43.168607+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:43.216769+0000 mon.a (mon.0) 151 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.217599+0000 mon.a (mon.0) 152 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.227086+0000 mon.a (mon.0) 153 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.244582+0000 mon.a (mon.0) 154 : cluster 0 fsmap 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.244639+0000 mon.a (mon.0) 155 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.244938+0000 mon.a (mon.0) 156 : cluster 0 mgrmap e13: y(active, since 76s) 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: cluster 2023-12-07T17:06:43.253010+0000 mon.a (mon.0) 157 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:43.257787+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:43.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:43 smithi184 bash[31336]: audit 2023-12-07T17:06:43.263853+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:38.168389+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:38.171396+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:38.171723+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:38.196879+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:38.302363+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:39.167691+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:40.168153+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:40.169519+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:40.302778+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:41.168088+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:42.168104+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:42.204613+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:42.303174+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:43.168607+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:43.216769+0000 mon.a (mon.0) 151 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.217599+0000 mon.a (mon.0) 152 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.227086+0000 mon.a (mon.0) 153 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.244582+0000 mon.a (mon.0) 154 : cluster 0 fsmap 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.244639+0000 mon.a (mon.0) 155 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.244938+0000 mon.a (mon.0) 156 : cluster 0 mgrmap e13: y(active, since 76s) 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: cluster 2023-12-07T17:06:43.253010+0000 mon.a (mon.0) 157 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:43.257787+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:43.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:43 smithi064 bash[25056]: audit 2023-12-07T17:06:43.263853+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:44.212151+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:49.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:44.212553+0000 mon.a (mon.0) 166 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:44.212621+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:44.214424+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:44.214679+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:44.303556+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:45.205276+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:46.205574+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:46.304032+0000 mgr.y (mgr.14152) 56 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:47.205648+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:48.205803+0000 mon.a (mon.0) 172 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:48.304436+0000 mgr.y (mgr.14152) 57 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.205972+0000 mon.a (mon.0) 173 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.214516+0000 mon.a (mon.0) 174 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:49.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.220766+0000 mon.a (mon.0) 175 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.240570+0000 mon.a (mon.0) 176 : cluster 0 fsmap 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.240616+0000 mon.a (mon.0) 177 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.240889+0000 mon.a (mon.0) 178 : cluster 0 mgrmap e13: y(active, since 82s) 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.241253+0000 mon.a (mon.0) 179 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.248295+0000 mon.a (mon.0) 180 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.248357+0000 mon.a (mon.0) 181 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: cluster 2023-12-07T17:06:49.248400+0000 mon.a (mon.0) 182 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] is down (out of quorum) 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.251397+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.255105+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.258631+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.261784+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:49 smithi184 bash[31336]: audit 2023-12-07T17:06:49.265704+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:44.212151+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:44.212553+0000 mon.a (mon.0) 166 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:44.212621+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:44.214424+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:44.214679+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:44.303556+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:45.205276+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:46.205574+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:46.304032+0000 mgr.y (mgr.14152) 56 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:47.205648+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:48.205803+0000 mon.a (mon.0) 172 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:48.304436+0000 mgr.y (mgr.14152) 57 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:49.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.205972+0000 mon.a (mon.0) 173 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.214516+0000 mon.a (mon.0) 174 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.220766+0000 mon.a (mon.0) 175 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.240570+0000 mon.a (mon.0) 176 : cluster 0 fsmap 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.240616+0000 mon.a (mon.0) 177 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.240889+0000 mon.a (mon.0) 178 : cluster 0 mgrmap e13: y(active, since 82s) 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.241253+0000 mon.a (mon.0) 179 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.248295+0000 mon.a (mon.0) 180 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.248357+0000 mon.a (mon.0) 181 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: cluster 2023-12-07T17:06:49.248400+0000 mon.a (mon.0) 182 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] is down (out of quorum) 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.251397+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.255105+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.258631+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.261784+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:49 smithi064 bash[25056]: audit 2023-12-07T17:06:49.265704+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:49.797 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-12-07T17:06:49.798 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"7ae18f76-9522-11ee-95a2-87774f69a715","modified":"2023-12-07T17:06:44.206067Z","created":"2023-12-07T17:04:09.272348Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3301","nonce":0},{"type":"v1","addr":"172.21.15.64:6790","nonce":0}]},"addr":"172.21.15.64:6790/0","public_addr":"172.21.15.64:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-07T17:06:49.801 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2023-12-07T17:06:50.501 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-07T17:06:50.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:49.800398+0000 mon.a (mon.0) 188 : audit 0 from='client.? 172.21.15.184:0/3208329215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:49.888938+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:49.890875+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.205618+0000 mon.a (mon.0) 191 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.232054+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.236779+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.240234+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.245068+0000 mon.a (mon.0) 195 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:50 smithi184 bash[31336]: audit 2023-12-07T17:06:50.250869+0000 mon.a (mon.0) 196 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:49.800398+0000 mon.a (mon.0) 188 : audit 0 from='client.? 172.21.15.184:0/3208329215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T17:06:50.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:49.888938+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:50.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:49.890875+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:50.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.205618+0000 mon.a (mon.0) 191 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:50.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.232054+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.236779+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.240234+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.245068+0000 mon.a (mon.0) 195 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:50.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:50 smithi064 bash[25056]: audit 2023-12-07T17:06:50.250869+0000 mon.a (mon.0) 196 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:38.168389+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:38.171396+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:38.171723+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:38.196879+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:38.302363+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:39.167691+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.515 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:40.168153+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:40.169519+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:40.302778+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:41.168088+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:42.168104+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:42.204613+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:42.303174+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:43.168607+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:43.216769+0000 mon.a (mon.0) 151 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.217599+0000 mon.a (mon.0) 152 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.227086+0000 mon.a (mon.0) 153 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.244582+0000 mon.a (mon.0) 154 : cluster 0 fsmap 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.244639+0000 mon.a (mon.0) 155 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.244938+0000 mon.a (mon.0) 156 : cluster 0 mgrmap e13: y(active, since 76s) 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:43.253010+0000 mon.a (mon.0) 157 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:43.257787+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.516 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:43.263853+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:44.212151+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:44.212553+0000 mon.a (mon.0) 166 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:44.212621+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:44.214424+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:44.214679+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:44.303556+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:45.205276+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:46.205574+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:46.304032+0000 mgr.y (mgr.14152) 56 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:47.205648+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:48.205803+0000 mon.a (mon.0) 172 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:48.304436+0000 mgr.y (mgr.14152) 57 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.205972+0000 mon.a (mon.0) 173 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.214516+0000 mon.a (mon.0) 174 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.220766+0000 mon.a (mon.0) 175 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:51.517 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.240570+0000 mon.a (mon.0) 176 : cluster 0 fsmap 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.240616+0000 mon.a (mon.0) 177 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.240889+0000 mon.a (mon.0) 178 : cluster 0 mgrmap e13: y(active, since 82s) 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.241253+0000 mon.a (mon.0) 179 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.248295+0000 mon.a (mon.0) 180 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.248357+0000 mon.a (mon.0) 181 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:49.248400+0000 mon.a (mon.0) 182 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] is down (out of quorum) 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.251397+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.255105+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.258631+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.261784+0000 mon.a (mon.0) 186 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.265704+0000 mon.a (mon.0) 187 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.800398+0000 mon.a (mon.0) 188 : audit 0 from='client.? 172.21.15.184:0/3208329215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.888938+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:49.890875+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:51.518 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.205618+0000 mon.a (mon.0) 191 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.232054+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.236779+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.240234+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.245068+0000 mon.a (mon.0) 195 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: audit 2023-12-07T17:06:50.250869+0000 mon.a (mon.0) 196 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:46.208070+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.240577+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.240703+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.241521+0000 mon.a (mon.0) 205 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:51.519 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.266526+0000 mon.a (mon.0) 206 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:46.208070+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.240577+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.240703+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.241521+0000 mon.a (mon.0) 205 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.266526+0000 mon.a (mon.0) 206 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.275029+0000 mon.a (mon.0) 207 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.275506+0000 mon.a (mon.0) 208 : cluster 0 fsmap 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.275541+0000 mon.a (mon.0) 209 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.275734+0000 mon.a (mon.0) 210 : cluster 0 mgrmap e13: y(active, since 84s) 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.275996+0000 mon.a (mon.0) 211 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2023-12-07T17:06:51.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.276029+0000 mon.a (mon.0) 212 : cluster 1 Cluster is now healthy 2023-12-07T17:06:51.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:51 smithi184 bash[31336]: cluster 2023-12-07T17:06:51.282845+0000 mon.a (mon.0) 213 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:46.208070+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.240577+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.240703+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.241521+0000 mon.a (mon.0) 205 : cluster 1 mon.a calling monitor election 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.266526+0000 mon.a (mon.0) 206 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.275029+0000 mon.a (mon.0) 207 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.275506+0000 mon.a (mon.0) 208 : cluster 0 fsmap 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.275541+0000 mon.a (mon.0) 209 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.275734+0000 mon.a (mon.0) 210 : cluster 0 mgrmap e13: y(active, since 84s) 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.275996+0000 mon.a (mon.0) 211 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2023-12-07T17:06:51.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.276029+0000 mon.a (mon.0) 212 : cluster 1 Cluster is now healthy 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[25056]: cluster 2023-12-07T17:06:51.282845+0000 mon.a (mon.0) 213 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.275029+0000 mon.a (mon.0) 207 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.275506+0000 mon.a (mon.0) 208 : cluster 0 fsmap 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.275541+0000 mon.a (mon.0) 209 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.275734+0000 mon.a (mon.0) 210 : cluster 0 mgrmap e13: y(active, since 84s) 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.275996+0000 mon.a (mon.0) 211 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.276029+0000 mon.a (mon.0) 212 : cluster 1 Cluster is now healthy 2023-12-07T17:06:51.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:51 smithi064 bash[36207]: cluster 2023-12-07T17:06:51.282845+0000 mon.a (mon.0) 213 : cluster 1 overall HEALTH_OK 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.205357+0000 mon.a (mon.0) 214 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: cluster 2023-12-07T17:06:52.305315+0000 mgr.y (mgr.14152) 65 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.653874+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.659481+0000 mon.a (mon.0) 216 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.661446+0000 mon.a (mon.0) 217 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.662669+0000 mon.a (mon.0) 218 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[25056]: audit 2023-12-07T17:06:52.663839+0000 mon.a (mon.0) 219 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.205357+0000 mon.a (mon.0) 214 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: cluster 2023-12-07T17:06:52.305315+0000 mgr.y (mgr.14152) 65 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.653874+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.659481+0000 mon.a (mon.0) 216 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.661446+0000 mon.a (mon.0) 217 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:53.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.662669+0000 mon.a (mon.0) 218 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:53.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:53 smithi064 bash[36207]: audit 2023-12-07T17:06:52.663839+0000 mon.a (mon.0) 219 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.205357+0000 mon.a (mon.0) 214 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: cluster 2023-12-07T17:06:52.305315+0000 mgr.y (mgr.14152) 65 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.653874+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.659481+0000 mon.a (mon.0) 216 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.661446+0000 mon.a (mon.0) 217 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.662669+0000 mon.a (mon.0) 218 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:53.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:53 smithi184 bash[31336]: audit 2023-12-07T17:06:52.663839+0000 mon.a (mon.0) 219 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:54.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:54 smithi184 bash[31336]: cephadm 2023-12-07T17:06:52.660943+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2023-12-07T17:06:54.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:54 smithi184 bash[31336]: cephadm 2023-12-07T17:06:52.665190+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring daemon mon.c on smithi064 2023-12-07T17:06:54.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:54 smithi064 bash[25056]: cephadm 2023-12-07T17:06:52.660943+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2023-12-07T17:06:54.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:54 smithi064 bash[25056]: cephadm 2023-12-07T17:06:52.665190+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring daemon mon.c on smithi064 2023-12-07T17:06:54.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:54 smithi064 bash[36207]: cephadm 2023-12-07T17:06:52.660943+0000 mgr.y (mgr.14152) 66 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2023-12-07T17:06:54.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:54 smithi064 bash[36207]: cephadm 2023-12-07T17:06:52.665190+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring daemon mon.c on smithi064 2023-12-07T17:06:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: cluster 2023-12-07T17:06:54.305735+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: audit 2023-12-07T17:06:54.579256+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: audit 2023-12-07T17:06:54.585056+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: audit 2023-12-07T17:06:54.634391+0000 mon.a (mon.0) 222 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: audit 2023-12-07T17:06:54.636028+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[25056]: audit 2023-12-07T17:06:54.637617+0000 mon.a (mon.0) 224 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:55.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: cluster 2023-12-07T17:06:54.305735+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:55.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: audit 2023-12-07T17:06:54.579256+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: audit 2023-12-07T17:06:54.585056+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: audit 2023-12-07T17:06:54.634391+0000 mon.a (mon.0) 222 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:55.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: audit 2023-12-07T17:06:54.636028+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:55.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:55 smithi064 bash[36207]: audit 2023-12-07T17:06:54.637617+0000 mon.a (mon.0) 224 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:55.954 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: cluster 2023-12-07T17:06:54.305735+0000 mgr.y (mgr.14152) 68 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:55.954 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: audit 2023-12-07T17:06:54.579256+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.955 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: audit 2023-12-07T17:06:54.585056+0000 mon.a (mon.0) 221 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:55.955 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: audit 2023-12-07T17:06:54.634391+0000 mon.a (mon.0) 222 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T17:06:55.955 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: audit 2023-12-07T17:06:54.636028+0000 mon.a (mon.0) 223 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T17:06:55.955 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:55 smithi184 bash[31336]: audit 2023-12-07T17:06:54.637617+0000 mon.a (mon.0) 224 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: cephadm 2023-12-07T17:06:54.633759+0000 mgr.y (mgr.14152) 69 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: cephadm 2023-12-07T17:06:54.639310+0000 mgr.y (mgr.14152) 70 : cephadm 1 Reconfiguring daemon mon.b on smithi184 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.546527+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.554577+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.558528+0000 mon.a (mon.0) 227 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.560661+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.561714+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:56.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[25056]: audit 2023-12-07T17:06:56.569992+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: cephadm 2023-12-07T17:06:54.633759+0000 mgr.y (mgr.14152) 69 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: cephadm 2023-12-07T17:06:54.639310+0000 mgr.y (mgr.14152) 70 : cephadm 1 Reconfiguring daemon mon.b on smithi184 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.546527+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.554577+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.558528+0000 mon.a (mon.0) 227 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.560661+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.561714+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:56.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:56 smithi064 bash[36207]: audit 2023-12-07T17:06:56.569992+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: cephadm 2023-12-07T17:06:54.633759+0000 mgr.y (mgr.14152) 69 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: cephadm 2023-12-07T17:06:54.639310+0000 mgr.y (mgr.14152) 70 : cephadm 1 Reconfiguring daemon mon.b on smithi184 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.546527+0000 mon.a (mon.0) 225 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.554577+0000 mon.a (mon.0) 226 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.558528+0000 mon.a (mon.0) 227 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.560661+0000 mon.a (mon.0) 228 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.561714+0000 mon.a (mon.0) 229 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:06:57.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:56 smithi184 bash[31336]: audit 2023-12-07T17:06:56.569992+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:06:57.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:57 smithi064 bash[25056]: cluster 2023-12-07T17:06:56.306206+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:57.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:57 smithi064 bash[36207]: cluster 2023-12-07T17:06:56.306206+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:57.997 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:06:57.997 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-12-07T17:06:57.997 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:06:57.997 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] [v2:172.21.15.64:3301/0,v1:172.21.15.64:6790/0] 2023-12-07T17:06:58.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:57 smithi184 bash[31336]: cluster 2023-12-07T17:06:56.306206+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:58.699 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-07T17:06:58.699 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:06:58.699 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2023-12-07T17:06:58.715 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:06:58.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:06:58.770 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:06:58.770 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2023-12-07T17:06:58.787 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:06:58.787 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:06:58.845 INFO:tasks.cephadm:Adding mgr.y on smithi064 2023-12-07T17:06:58.845 INFO:tasks.cephadm:Adding mgr.x on smithi184 2023-12-07T17:06:58.845 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply mgr '2;smithi064=y;smithi184=x' 2023-12-07T17:06:58.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:58 smithi064 bash[25056]: audit 2023-12-07T17:06:58.000233+0000 mon.a (mon.0) 231 : audit 0 from='client.? 172.21.15.64:0/2434551784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:58 smithi064 bash[36207]: audit 2023-12-07T17:06:58.000233+0000 mon.a (mon.0) 231 : audit 0 from='client.? 172.21.15.64:0/2434551784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:58.899 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:58 smithi184 bash[31336]: audit 2023-12-07T17:06:58.000233+0000 mon.a (mon.0) 231 : audit 0 from='client.? 172.21.15.64:0/2434551784' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:06:59.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:06:59 smithi064 bash[25056]: cluster 2023-12-07T17:06:58.306670+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:06:59.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:06:59 smithi064 bash[36207]: cluster 2023-12-07T17:06:58.306670+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:00.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:06:59 smithi184 bash[31336]: cluster 2023-12-07T17:06:58.306670+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:01.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:01 smithi064 bash[25056]: cluster 2023-12-07T17:07:00.307112+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:01.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:01 smithi064 bash[36207]: cluster 2023-12-07T17:07:00.307112+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:02.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:01 smithi184 bash[31336]: cluster 2023-12-07T17:07:00.307112+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:02.595 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:07:03.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:02 smithi184 bash[31336]: cluster 2023-12-07T17:07:02.307596+0000 mgr.y (mgr.14152) 74 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:03.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:02 smithi064 bash[25056]: cluster 2023-12-07T17:07:02.307596+0000 mgr.y (mgr.14152) 74 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:03.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:02 smithi064 bash[36207]: cluster 2023-12-07T17:07:02.307596+0000 mgr.y (mgr.14152) 74 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:05.267 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: cluster 2023-12-07T17:07:04.308109+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.269040+0000 mon.a (mon.0) 232 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.274458+0000 mon.a (mon.0) 233 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.277425+0000 mon.a (mon.0) 234 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.278901+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.288367+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.290775+0000 mon.a (mon.0) 237 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.295095+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.298229+0000 mon.a (mon.0) 239 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[36207]: audit 2023-12-07T17:07:05.299556+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: cluster 2023-12-07T17:07:04.308109+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.269040+0000 mon.a (mon.0) 232 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.274458+0000 mon.a (mon.0) 233 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.277425+0000 mon.a (mon.0) 234 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.278901+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.288367+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.290775+0000 mon.a (mon.0) 237 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.295095+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T17:07:05.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.298229+0000 mon.a (mon.0) 239 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:05.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:05 smithi064 bash[25056]: audit 2023-12-07T17:07:05.299556+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:05.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: cluster 2023-12-07T17:07:04.308109+0000 mgr.y (mgr.14152) 75 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.269040+0000 mon.a (mon.0) 232 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.274458+0000 mon.a (mon.0) 233 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.277425+0000 mon.a (mon.0) 234 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.278901+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.288367+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.290775+0000 mon.a (mon.0) 237 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.295095+0000 mon.a (mon.0) 238 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.298229+0000 mon.a (mon.0) 239 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:05.754 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:05 smithi184 bash[31336]: audit 2023-12-07T17:07:05.299556+0000 mon.a (mon.0) 240 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:06.159 DEBUG:teuthology.orchestra.run.smithi184:mgr.x> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.x.service 2023-12-07T17:07:06.205 INFO:tasks.cephadm:Deploying OSDs... 2023-12-07T17:07:06.205 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:07:06.205 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:07:06.213 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-12-07T17:07:06.213 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/nvme1n1 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/nvme1n1 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 522 Links: 1 Device type: 103,2 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:05:42.485413172 +0000 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:02:59.819231109 +0000 2023-12-07T17:07:06.263 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:02:59.819231109 +0000 2023-12-07T17:07:06.264 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:07:06.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2023-12-07T17:07:06.318 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:07:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:07:06.319 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000446382 s, 1.1 MB/s 2023-12-07T17:07:06.320 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2023-12-07T17:07:06.372 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/nvme2n1 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/nvme2n1 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 530 Links: 1 Device type: 103,4 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:05:42.505412709 +0000 2023-12-07T17:07:06.426 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:02:59.939288342 +0000 2023-12-07T17:07:06.427 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:02:59.939288342 +0000 2023-12-07T17:07:06.427 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:07:06.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2023-12-07T17:07:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:07:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:07:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000453556 s, 1.1 MB/s 2023-12-07T17:07:06.483 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2023-12-07T17:07:06.536 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/nvme3n1 2023-12-07T17:07:06.587 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/nvme3n1 2023-12-07T17:07:06.587 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 538 Links: 1 Device type: 103,6 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:05:42.521412342 +0000 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:03:00.063347482 +0000 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:03:00.063347482 +0000 2023-12-07T17:07:06.588 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:07:06.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2023-12-07T17:07:06.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[25056]: audit 2023-12-07T17:07:05.256981+0000 mgr.y (mgr.14152) 76 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi064=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:06.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[25056]: cephadm 2023-12-07T17:07:05.260680+0000 mgr.y (mgr.14152) 77 : cephadm 1 Saving service mgr spec with placement smithi064=y;smithi184=x;count:2 2023-12-07T17:07:06.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[25056]: cephadm 2023-12-07T17:07:05.301511+0000 mgr.y (mgr.14152) 78 : cephadm 1 Deploying daemon mgr.x on smithi184 2023-12-07T17:07:06.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[36207]: audit 2023-12-07T17:07:05.256981+0000 mgr.y (mgr.14152) 76 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi064=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:06.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[36207]: cephadm 2023-12-07T17:07:05.260680+0000 mgr.y (mgr.14152) 77 : cephadm 1 Saving service mgr spec with placement smithi064=y;smithi184=x;count:2 2023-12-07T17:07:06.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:06 smithi064 bash[36207]: cephadm 2023-12-07T17:07:05.301511+0000 mgr.y (mgr.14152) 78 : cephadm 1 Deploying daemon mgr.x on smithi184 2023-12-07T17:07:06.622 INFO:journalctl@ceph.mgr.x.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:07:06.629 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:07:06.629 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:07:06.629 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000484119 s, 1.1 MB/s 2023-12-07T17:07:06.630 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2023-12-07T17:07:06.681 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/nvme4n1 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/nvme4n1 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 546 Links: 1 Device type: 103,8 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-12-07 17:05:42.541411879 +0000 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-12-07 17:03:00.179402807 +0000 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-12-07 17:03:00.179402807 +0000 2023-12-07T17:07:06.731 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-12-07T17:07:06.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2023-12-07T17:07:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-12-07T17:07:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-12-07T17:07:06.787 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000463975 s, 1.1 MB/s 2023-12-07T17:07:06.787 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2023-12-07T17:07:06.798 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:06 smithi184 bash[31336]: audit 2023-12-07T17:07:05.256981+0000 mgr.y (mgr.14152) 76 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi064=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:06.798 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:06 smithi184 bash[31336]: cephadm 2023-12-07T17:07:05.260680+0000 mgr.y (mgr.14152) 77 : cephadm 1 Saving service mgr spec with placement smithi064=y;smithi184=x;count:2 2023-12-07T17:07:06.798 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:06 smithi184 bash[31336]: cephadm 2023-12-07T17:07:05.301511+0000 mgr.y (mgr.14152) 78 : cephadm 1 Deploying daemon mgr.x on smithi184 2023-12-07T17:07:06.841 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:07:06.841 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T17:07:06.847 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-12-07T17:07:06.848 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/nvme1n1 2023-12-07T17:07:06.898 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/nvme1n1 2023-12-07T17:07:06.898 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:06.898 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 522 Links: 1 Device type: 103,2 2023-12-07T17:07:06.898 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:06.898 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:06:17.538552714 +0000 2023-12-07T17:07:06.899 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:03:01.203075401 +0000 2023-12-07T17:07:06.899 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:03:01.203075401 +0000 2023-12-07T17:07:06.899 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:07:06.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2023-12-07T17:07:06.958 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:07:06.958 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:07:06.958 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000478461 s, 1.1 MB/s 2023-12-07T17:07:06.959 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2023-12-07T17:07:07.011 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/nvme2n1 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/nvme2n1 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 530 Links: 1 Device type: 103,4 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:06:17.558552292 +0000 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:03:01.323072635 +0000 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:03:01.323072635 +0000 2023-12-07T17:07:07.063 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:07:07.063 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2023-12-07T17:07:07.116 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:07:07.117 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:07:07.117 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000468518 s, 1.1 MB/s 2023-12-07T17:07:07.118 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2023-12-07T17:07:07.168 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/nvme3n1 2023-12-07T17:07:07.217 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/nvme3n1 2023-12-07T17:07:07.217 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 538 Links: 1 Device type: 103,6 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:06:17.578551873 +0000 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:03:01.439069959 +0000 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:03:01.439069959 +0000 2023-12-07T17:07:07.218 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:07:07.219 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2023-12-07T17:07:07.270 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:07:07.270 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:07:07.270 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00047301 s, 1.1 MB/s 2023-12-07T17:07:07.271 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2023-12-07T17:07:07.324 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/nvme4n1 2023-12-07T17:07:07.378 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/nvme4n1 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 546 Links: 1 Device type: 103,8 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-12-07 17:06:17.602551367 +0000 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-12-07 17:03:01.559067191 +0000 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-12-07 17:03:01.559067191 +0000 2023-12-07T17:07:07.379 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-12-07T17:07:07.379 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2023-12-07T17:07:07.433 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-12-07T17:07:07.433 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-12-07T17:07:07.434 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000310976 s, 1.6 MB/s 2023-12-07T17:07:07.435 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2023-12-07T17:07:07.482 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/nvme4n1... 2023-12-07T17:07:07.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2023-12-07T17:07:07.705 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:07 smithi184 bash[31336]: cluster 2023-12-07T17:07:06.308579+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:07.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:07 smithi064 bash[25056]: cluster 2023-12-07T17:07:06.308579+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:07.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:07 smithi064 bash[36207]: cluster 2023-12-07T17:07:06.308579+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:08.002 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:07 smithi184 systemd[1]: Started Ceph mgr.x for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: audit 2023-12-07T17:07:07.988782+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: audit 2023-12-07T17:07:07.994820+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: audit 2023-12-07T17:07:08.028157+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: audit 2023-12-07T17:07:08.034461+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: audit 2023-12-07T17:07:08.101019+0000 mon.a (mon.0) 245 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:09.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:08 smithi184 bash[31336]: cluster 2023-12-07T17:07:08.308990+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: audit 2023-12-07T17:07:07.988782+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: audit 2023-12-07T17:07:07.994820+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: audit 2023-12-07T17:07:08.028157+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: audit 2023-12-07T17:07:08.034461+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: audit 2023-12-07T17:07:08.101019+0000 mon.a (mon.0) 245 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:09.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[25056]: cluster 2023-12-07T17:07:08.308990+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: audit 2023-12-07T17:07:07.988782+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: audit 2023-12-07T17:07:07.994820+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: audit 2023-12-07T17:07:08.028157+0000 mon.a (mon.0) 243 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: audit 2023-12-07T17:07:08.034461+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: audit 2023-12-07T17:07:08.101019+0000 mon.a (mon.0) 245 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:09.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:08 smithi064 bash[36207]: cluster 2023-12-07T17:07:08.308990+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:09.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:09 smithi184 bash[32290]: debug 2023-12-07T17:07:09.073+0000 7f3496fd8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:07:10.255 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:09 smithi184 bash[32290]: debug 2023-12-07T17:07:09.973+0000 7f3496fd8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:07:10.255 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[32290]: debug 2023-12-07T17:07:10.077+0000 7f3496fd8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:07:10.255 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[32290]: debug 2023-12-07T17:07:10.177+0000 7f3496fd8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:07:10.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[31336]: audit 2023-12-07T17:07:09.259370+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:10.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[32290]: debug 2023-12-07T17:07:10.441+0000 7f3496fd8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:07:10.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:10 smithi064 bash[25056]: audit 2023-12-07T17:07:09.259370+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:10.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:10 smithi064 bash[36207]: audit 2023-12-07T17:07:09.259370+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:11.063 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[32290]: debug 2023-12-07T17:07:10.793+0000 7f3496fd8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:07:11.063 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:10 smithi184 bash[32290]: debug 2023-12-07T17:07:10.949+0000 7f3496fd8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:07:11.518 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:11 smithi184 bash[32290]: debug 2023-12-07T17:07:11.161+0000 7f3496fd8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:07:11.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:11 smithi184 bash[31336]: cluster 2023-12-07T17:07:10.309425+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:11.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:11 smithi064 bash[25056]: cluster 2023-12-07T17:07:10.309425+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:11.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:11 smithi064 bash[36207]: cluster 2023-12-07T17:07:10.309425+0000 mgr.y (mgr.14152) 81 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:12.227 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:07:12.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:12 smithi184 bash[32290]: debug 2023-12-07T17:07:12.173+0000 7f3496fd8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:07:12.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:12 smithi184 bash[32290]: debug 2023-12-07T17:07:12.285+0000 7f3496fd8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:07:12.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:12 smithi184 bash[32290]: debug 2023-12-07T17:07:12.401+0000 7f3496fd8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:07:13.313 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:12 smithi184 bash[32290]: debug 2023-12-07T17:07:12.877+0000 7f3496fd8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:07:13.313 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:13 smithi184 bash[32290]: debug 2023-12-07T17:07:13.105+0000 7f3496fd8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:07:13.813 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:13 smithi184 bash[32290]: debug 2023-12-07T17:07:13.573+0000 7f3496fd8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:07:13.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:13 smithi184 bash[31336]: cluster 2023-12-07T17:07:12.309855+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:13.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:13 smithi184 bash[31336]: audit 2023-12-07T17:07:13.389168+0000 mon.a (mon.0) 247 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:13.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:13 smithi184 bash[31336]: audit 2023-12-07T17:07:13.396241+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[25056]: cluster 2023-12-07T17:07:12.309855+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[25056]: audit 2023-12-07T17:07:13.389168+0000 mon.a (mon.0) 247 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[25056]: audit 2023-12-07T17:07:13.396241+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[36207]: cluster 2023-12-07T17:07:12.309855+0000 mgr.y (mgr.14152) 82 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[36207]: audit 2023-12-07T17:07:13.389168+0000 mon.a (mon.0) 247 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:13.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:13 smithi064 bash[36207]: audit 2023-12-07T17:07:13.396241+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:14.527 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[32290]: debug 2023-12-07T17:07:14.089+0000 7f3496fd8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[32290]: debug 2023-12-07T17:07:14.641+0000 7f3496fd8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[32290]: debug 2023-12-07T17:07:14.749+0000 7f3496fd8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.021774+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.022833+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.029023+0000 mon.a (mon.0) 251 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.095999+0000 mon.a (mon.0) 252 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:14.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.097489+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:14.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:14 smithi184 bash[31336]: audit 2023-12-07T17:07:14.098536+0000 mon.a (mon.0) 254 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.021774+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.022833+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.029023+0000 mon.a (mon.0) 251 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.095999+0000 mon.a (mon.0) 252 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.097489+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[25056]: audit 2023-12-07T17:07:14.098536+0000 mon.a (mon.0) 254 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.021774+0000 mon.a (mon.0) 249 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.022833+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.029023+0000 mon.a (mon.0) 251 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:14.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.095999+0000 mon.a (mon.0) 252 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T17:07:14.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.097489+0000 mon.a (mon.0) 253 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T17:07:14.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:14 smithi064 bash[36207]: audit 2023-12-07T17:07:14.098536+0000 mon.a (mon.0) 254 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:15.482 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.121+0000 7f3496fd8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:07:15.482 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.229+0000 7f3496fd8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:07:15.482 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.369+0000 7f3496fd8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[25056]: cephadm 2023-12-07T17:07:14.095478+0000 mgr.y (mgr.14152) 83 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[25056]: cephadm 2023-12-07T17:07:14.099691+0000 mgr.y (mgr.14152) 84 : cephadm 1 Reconfiguring daemon mgr.y on smithi064 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[25056]: cluster 2023-12-07T17:07:14.310268+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[36207]: cephadm 2023-12-07T17:07:14.095478+0000 mgr.y (mgr.14152) 83 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[36207]: cephadm 2023-12-07T17:07:14.099691+0000 mgr.y (mgr.14152) 84 : cephadm 1 Reconfiguring daemon mgr.y on smithi064 2023-12-07T17:07:15.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:15 smithi064 bash[36207]: cluster 2023-12-07T17:07:14.310268+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:15.794 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.481+0000 7f3496fd8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:07:15.794 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.605+0000 7f3496fd8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:07:15.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[31336]: cephadm 2023-12-07T17:07:14.095478+0000 mgr.y (mgr.14152) 83 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2023-12-07T17:07:15.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[31336]: cephadm 2023-12-07T17:07:14.099691+0000 mgr.y (mgr.14152) 84 : cephadm 1 Reconfiguring daemon mgr.y on smithi064 2023-12-07T17:07:15.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[31336]: cluster 2023-12-07T17:07:14.310268+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:16.063 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.793+0000 7f3496fd8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:07:16.063 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:15 smithi184 bash[32290]: debug 2023-12-07T17:07:15.917+0000 7f3496fd8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:07:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/nvme4n1 2023-12-07T17:07:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:07:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2023-12-07T17:07:16.096 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-12-07T17:07:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-12-07T17:07:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149122 s, 703 MB/s 2023-12-07T17:07:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-12-07T17:07:16.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi064:/dev/nvme4n1 2023-12-07T17:07:17.563 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[32290]: debug 2023-12-07T17:07:17.153+0000 7f3496fd8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:07:17.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: cluster 2023-12-07T17:07:16.310568+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:17.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.614725+0000 mon.a (mon.0) 255 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.622320+0000 mon.a (mon.0) 256 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.669108+0000 mon.a (mon.0) 257 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:17.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.671897+0000 mon.a (mon.0) 258 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.673239+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:16.679941+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: cluster 2023-12-07T17:07:17.178586+0000 mon.a (mon.0) 261 : cluster 0 Standby manager daemon x started 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:17.181291+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:17.182272+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:17.184347+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[25056]: audit 2023-12-07T17:07:17.184991+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: cluster 2023-12-07T17:07:16.310568+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.614725+0000 mon.a (mon.0) 255 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.622320+0000 mon.a (mon.0) 256 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.669108+0000 mon.a (mon.0) 257 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.671897+0000 mon.a (mon.0) 258 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.673239+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:16.679941+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: cluster 2023-12-07T17:07:17.178586+0000 mon.a (mon.0) 261 : cluster 0 Standby manager daemon x started 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:17.181291+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:17.182272+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:17.184347+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:07:17.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:17 smithi064 bash[36207]: audit 2023-12-07T17:07:17.184991+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: cluster 2023-12-07T17:07:16.310568+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.614725+0000 mon.a (mon.0) 255 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.622320+0000 mon.a (mon.0) 256 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.669108+0000 mon.a (mon.0) 257 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.671897+0000 mon.a (mon.0) 258 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.673239+0000 mon.a (mon.0) 259 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:16.679941+0000 mon.a (mon.0) 260 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: cluster 2023-12-07T17:07:17.178586+0000 mon.a (mon.0) 261 : cluster 0 Standby manager daemon x started 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:17.181291+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:17.182272+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:17.184347+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:07:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:17 smithi184 bash[31336]: audit 2023-12-07T17:07:17.184991+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.184:0/1040536058' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:07:19.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:18 smithi184 bash[31336]: cluster 2023-12-07T17:07:17.689578+0000 mon.a (mon.0) 262 : cluster 0 mgrmap e14: y(active, since 111s), standbys: x 2023-12-07T17:07:19.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:18 smithi184 bash[31336]: audit 2023-12-07T17:07:17.689826+0000 mon.a (mon.0) 263 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:07:19.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:18 smithi184 bash[31336]: cluster 2023-12-07T17:07:18.310899+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:19.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[25056]: cluster 2023-12-07T17:07:17.689578+0000 mon.a (mon.0) 262 : cluster 0 mgrmap e14: y(active, since 111s), standbys: x 2023-12-07T17:07:19.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[25056]: audit 2023-12-07T17:07:17.689826+0000 mon.a (mon.0) 263 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:07:19.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[25056]: cluster 2023-12-07T17:07:18.310899+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:19.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[36207]: cluster 2023-12-07T17:07:17.689578+0000 mon.a (mon.0) 262 : cluster 0 mgrmap e14: y(active, since 111s), standbys: x 2023-12-07T17:07:19.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[36207]: audit 2023-12-07T17:07:17.689826+0000 mon.a (mon.0) 263 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:07:19.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:18 smithi064 bash[36207]: cluster 2023-12-07T17:07:18.310899+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:21.720 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:07:21.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:21 smithi184 bash[31336]: cluster 2023-12-07T17:07:20.311321+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:21.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:21 smithi064 bash[25056]: cluster 2023-12-07T17:07:20.311321+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:21.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:21 smithi064 bash[36207]: cluster 2023-12-07T17:07:20.311321+0000 mgr.y (mgr.14152) 88 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:23.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:23 smithi184 bash[31336]: cluster 2023-12-07T17:07:22.311670+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:23.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:23 smithi064 bash[25056]: cluster 2023-12-07T17:07:22.311670+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:23.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:23 smithi064 bash[36207]: cluster 2023-12-07T17:07:22.311670+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:25.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:25 smithi184 bash[31336]: cluster 2023-12-07T17:07:24.312018+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:25.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:25 smithi184 bash[31336]: audit 2023-12-07T17:07:24.599009+0000 mgr.y (mgr.14152) 91 : audit 0 from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:25.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:25 smithi184 bash[31336]: audit 2023-12-07T17:07:24.606352+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:07:25.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:25 smithi184 bash[31336]: audit 2023-12-07T17:07:24.612969+0000 mon.a (mon.0) 265 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:07:25.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:25 smithi184 bash[31336]: audit 2023-12-07T17:07:24.614351+0000 mon.a (mon.0) 266 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:25.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[25056]: cluster 2023-12-07T17:07:24.312018+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:25.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[25056]: audit 2023-12-07T17:07:24.599009+0000 mgr.y (mgr.14152) 91 : audit 0 from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:25.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[25056]: audit 2023-12-07T17:07:24.606352+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:07:25.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[25056]: audit 2023-12-07T17:07:24.612969+0000 mon.a (mon.0) 265 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:07:25.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[25056]: audit 2023-12-07T17:07:24.614351+0000 mon.a (mon.0) 266 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[36207]: cluster 2023-12-07T17:07:24.312018+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[36207]: audit 2023-12-07T17:07:24.599009+0000 mgr.y (mgr.14152) 91 : audit 0 from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:07:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[36207]: audit 2023-12-07T17:07:24.606352+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:07:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[36207]: audit 2023-12-07T17:07:24.612969+0000 mon.a (mon.0) 265 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:07:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:25 smithi064 bash[36207]: audit 2023-12-07T17:07:24.614351+0000 mon.a (mon.0) 266 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:27.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:27 smithi184 bash[31336]: cluster 2023-12-07T17:07:26.312464+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:27.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:27 smithi064 bash[25056]: cluster 2023-12-07T17:07:26.312464+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:27.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:27 smithi064 bash[36207]: cluster 2023-12-07T17:07:26.312464+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:29.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:29 smithi184 bash[31336]: cluster 2023-12-07T17:07:28.312894+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:29.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:29 smithi064 bash[36207]: cluster 2023-12-07T17:07:28.312894+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:29.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:29 smithi064 bash[25056]: cluster 2023-12-07T17:07:28.312894+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:31.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:31 smithi064 bash[25056]: cluster 2023-12-07T17:07:30.313303+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:31.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:31 smithi064 bash[36207]: cluster 2023-12-07T17:07:30.313303+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:31.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:31 smithi184 bash[31336]: cluster 2023-12-07T17:07:30.313303+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:33.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:33 smithi184 bash[31336]: cluster 2023-12-07T17:07:32.313676+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:33.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:33 smithi184 bash[31336]: audit 2023-12-07T17:07:33.175871+0000 mon.a (mon.0) 267 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]: dispatch 2023-12-07T17:07:33.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:33 smithi184 bash[31336]: audit 2023-12-07T17:07:33.178202+0000 mon.a (mon.0) 268 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]': finished 2023-12-07T17:07:33.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:33 smithi184 bash[31336]: cluster 2023-12-07T17:07:33.183882+0000 mon.a (mon.0) 269 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2023-12-07T17:07:33.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:33 smithi184 bash[31336]: audit 2023-12-07T17:07:33.184358+0000 mon.a (mon.0) 270 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:07:33.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[25056]: cluster 2023-12-07T17:07:32.313676+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:33.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[25056]: audit 2023-12-07T17:07:33.175871+0000 mon.a (mon.0) 267 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]: dispatch 2023-12-07T17:07:33.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[25056]: audit 2023-12-07T17:07:33.178202+0000 mon.a (mon.0) 268 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]': finished 2023-12-07T17:07:33.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[25056]: cluster 2023-12-07T17:07:33.183882+0000 mon.a (mon.0) 269 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2023-12-07T17:07:33.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[25056]: audit 2023-12-07T17:07:33.184358+0000 mon.a (mon.0) 270 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:07:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[36207]: cluster 2023-12-07T17:07:32.313676+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[36207]: audit 2023-12-07T17:07:33.175871+0000 mon.a (mon.0) 267 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]: dispatch 2023-12-07T17:07:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[36207]: audit 2023-12-07T17:07:33.178202+0000 mon.a (mon.0) 268 : audit 1 from='client.? 172.21.15.64:0/1806509159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb84ad8-fb61-45c8-99fa-eb8a522bcca9"}]': finished 2023-12-07T17:07:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[36207]: cluster 2023-12-07T17:07:33.183882+0000 mon.a (mon.0) 269 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2023-12-07T17:07:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:33 smithi064 bash[36207]: audit 2023-12-07T17:07:33.184358+0000 mon.a (mon.0) 270 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:07:35.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:35 smithi184 bash[31336]: cluster 2023-12-07T17:07:34.314176+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:35.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:35 smithi184 bash[31336]: audit 2023-12-07T17:07:34.891833+0000 mon.c (mon.2) 3 : audit 0 from='client.? 172.21.15.64:0/4172601444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:07:35.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:35 smithi064 bash[25056]: cluster 2023-12-07T17:07:34.314176+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:35.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:35 smithi064 bash[25056]: audit 2023-12-07T17:07:34.891833+0000 mon.c (mon.2) 3 : audit 0 from='client.? 172.21.15.64:0/4172601444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:07:35.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:35 smithi064 bash[36207]: cluster 2023-12-07T17:07:34.314176+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:35.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:35 smithi064 bash[36207]: audit 2023-12-07T17:07:34.891833+0000 mon.c (mon.2) 3 : audit 0 from='client.? 172.21.15.64:0/4172601444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:07:37.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:37 smithi184 bash[31336]: cluster 2023-12-07T17:07:36.314627+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:37.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:37 smithi064 bash[25056]: cluster 2023-12-07T17:07:36.314627+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:37.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:37 smithi064 bash[36207]: cluster 2023-12-07T17:07:36.314627+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:39.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:39 smithi184 bash[31336]: cluster 2023-12-07T17:07:38.315066+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:39.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:39 smithi064 bash[25056]: cluster 2023-12-07T17:07:38.315066+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:39.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:39 smithi064 bash[36207]: cluster 2023-12-07T17:07:38.315066+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:41.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:41 smithi064 bash[25056]: cluster 2023-12-07T17:07:40.315529+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:41.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:41 smithi064 bash[36207]: cluster 2023-12-07T17:07:40.315529+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:42.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:41 smithi184 bash[31336]: cluster 2023-12-07T17:07:40.315529+0000 mgr.y (mgr.14152) 99 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:43.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:43 smithi064 bash[36207]: cluster 2023-12-07T17:07:42.315949+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:43.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:43 smithi064 bash[25056]: cluster 2023-12-07T17:07:42.315949+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:44.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:43 smithi184 bash[31336]: cluster 2023-12-07T17:07:42.315949+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:45.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:45 smithi064 bash[25056]: cluster 2023-12-07T17:07:44.316411+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:45.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:45 smithi064 bash[36207]: cluster 2023-12-07T17:07:44.316411+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:46.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:45 smithi184 bash[31336]: cluster 2023-12-07T17:07:44.316411+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[36207]: cluster 2023-12-07T17:07:46.316840+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[36207]: audit 2023-12-07T17:07:47.051612+0000 mon.a (mon.0) 271 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T17:07:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[36207]: audit 2023-12-07T17:07:47.053114+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[25056]: cluster 2023-12-07T17:07:46.316840+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[25056]: audit 2023-12-07T17:07:47.051612+0000 mon.a (mon.0) 271 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T17:07:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:47 smithi064 bash[25056]: audit 2023-12-07T17:07:47.053114+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:48.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:47 smithi184 bash[31336]: cluster 2023-12-07T17:07:46.316840+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:48.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:47 smithi184 bash[31336]: audit 2023-12-07T17:07:47.051612+0000 mon.a (mon.0) 271 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T17:07:48.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:47 smithi184 bash[31336]: audit 2023-12-07T17:07:47.053114+0000 mon.a (mon.0) 272 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:48.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:48 smithi064 bash[25056]: cephadm 2023-12-07T17:07:47.054503+0000 mgr.y (mgr.14152) 103 : cephadm 1 Deploying daemon osd.0 on smithi064 2023-12-07T17:07:48.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:48 smithi064 bash[36207]: cephadm 2023-12-07T17:07:47.054503+0000 mgr.y (mgr.14152) 103 : cephadm 1 Deploying daemon osd.0 on smithi064 2023-12-07T17:07:49.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:48 smithi184 bash[31336]: cephadm 2023-12-07T17:07:47.054503+0000 mgr.y (mgr.14152) 103 : cephadm 1 Deploying daemon osd.0 on smithi064 2023-12-07T17:07:49.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:49 smithi064 bash[25056]: cluster 2023-12-07T17:07:48.317303+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:49.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:49 smithi064 bash[36207]: cluster 2023-12-07T17:07:48.317303+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:50.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:49 smithi184 bash[31336]: cluster 2023-12-07T17:07:48.317303+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:51.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:51 smithi064 bash[36207]: cluster 2023-12-07T17:07:50.317702+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:51.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:51 smithi064 bash[25056]: cluster 2023-12-07T17:07:50.317702+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:52.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:51 smithi184 bash[31336]: cluster 2023-12-07T17:07:50.317702+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:53.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[25056]: audit 2023-12-07T17:07:51.844935+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[25056]: audit 2023-12-07T17:07:51.851459+0000 mon.a (mon.0) 274 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[25056]: audit 2023-12-07T17:07:51.851966+0000 mon.a (mon.0) 275 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:53.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[25056]: cluster 2023-12-07T17:07:52.318105+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:53.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[36207]: audit 2023-12-07T17:07:51.844935+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[36207]: audit 2023-12-07T17:07:51.851459+0000 mon.a (mon.0) 274 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[36207]: audit 2023-12-07T17:07:51.851966+0000 mon.a (mon.0) 275 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:53.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:52 smithi064 bash[36207]: cluster 2023-12-07T17:07:52.318105+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:53.313 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:52 smithi184 bash[31336]: audit 2023-12-07T17:07:51.844935+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.313 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:52 smithi184 bash[31336]: audit 2023-12-07T17:07:51.851459+0000 mon.a (mon.0) 274 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:53.313 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:52 smithi184 bash[31336]: audit 2023-12-07T17:07:51.851966+0000 mon.a (mon.0) 275 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:07:53.313 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:52 smithi184 bash[31336]: cluster 2023-12-07T17:07:52.318105+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:55.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:55 smithi064 bash[25056]: cluster 2023-12-07T17:07:54.318627+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:55.692 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:55 smithi064 bash[36207]: cluster 2023-12-07T17:07:54.318627+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:55.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:55 smithi184 bash[31336]: cluster 2023-12-07T17:07:54.318627+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:57.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:57 smithi184 bash[31336]: cluster 2023-12-07T17:07:56.319033+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:57.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:57 smithi184 bash[31336]: audit 2023-12-07T17:07:57.240167+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:57.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:57 smithi184 bash[31336]: audit 2023-12-07T17:07:57.246061+0000 mon.a (mon.0) 277 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:57.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[25056]: cluster 2023-12-07T17:07:56.319033+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:57.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[25056]: audit 2023-12-07T17:07:57.240167+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:57.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[25056]: audit 2023-12-07T17:07:57.246061+0000 mon.a (mon.0) 277 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:57.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[36207]: cluster 2023-12-07T17:07:56.319033+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:57.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[36207]: audit 2023-12-07T17:07:57.240167+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:57.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:57 smithi064 bash[36207]: audit 2023-12-07T17:07:57.246061+0000 mon.a (mon.0) 277 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:58.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:58 smithi184 bash[31336]: audit 2023-12-07T17:07:57.868598+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:58.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:58 smithi184 bash[31336]: audit 2023-12-07T17:07:57.870072+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:58.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:58 smithi184 bash[31336]: audit 2023-12-07T17:07:57.878951+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:58.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[25056]: audit 2023-12-07T17:07:57.868598+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:58.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[25056]: audit 2023-12-07T17:07:57.870072+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:58.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[25056]: audit 2023-12-07T17:07:57.878951+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:58.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[36207]: audit 2023-12-07T17:07:57.868598+0000 mon.a (mon.0) 278 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:07:58.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[36207]: audit 2023-12-07T17:07:57.870072+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:07:58.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:58 smithi064 bash[36207]: audit 2023-12-07T17:07:57.878951+0000 mon.a (mon.0) 280 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:07:59.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:59 smithi184 bash[31336]: cluster 2023-12-07T17:07:58.319454+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:59.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:07:59 smithi184 bash[31336]: audit 2023-12-07T17:07:58.749510+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T17:07:59.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:59 smithi064 bash[25056]: cluster 2023-12-07T17:07:58.319454+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:59.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:07:59 smithi064 bash[25056]: audit 2023-12-07T17:07:58.749510+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T17:07:59.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:59 smithi064 bash[36207]: cluster 2023-12-07T17:07:58.319454+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:07:59.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:07:59 smithi064 bash[36207]: audit 2023-12-07T17:07:58.749510+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T17:08:00.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: audit 2023-12-07T17:07:59.542118+0000 mon.a (mon.0) 282 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: cluster 2023-12-07T17:07:59.548730+0000 mon.a (mon.0) 283 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: audit 2023-12-07T17:07:59.549545+0000 mon.a (mon.0) 284 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: audit 2023-12-07T17:07:59.550202+0000 mon.a (mon.0) 285 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: audit 2023-12-07T17:08:00.543564+0000 mon.a (mon.0) 286 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: cluster 2023-12-07T17:08:00.547459+0000 mon.a (mon.0) 287 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2023-12-07T17:08:00.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:00 smithi184 bash[31336]: audit 2023-12-07T17:08:00.548462+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:00.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: audit 2023-12-07T17:07:59.542118+0000 mon.a (mon.0) 282 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: cluster 2023-12-07T17:07:59.548730+0000 mon.a (mon.0) 283 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: audit 2023-12-07T17:07:59.549545+0000 mon.a (mon.0) 284 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: audit 2023-12-07T17:07:59.550202+0000 mon.a (mon.0) 285 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: audit 2023-12-07T17:08:00.543564+0000 mon.a (mon.0) 286 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: cluster 2023-12-07T17:08:00.547459+0000 mon.a (mon.0) 287 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[25056]: audit 2023-12-07T17:08:00.548462+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: audit 2023-12-07T17:07:59.542118+0000 mon.a (mon.0) 282 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: cluster 2023-12-07T17:07:59.548730+0000 mon.a (mon.0) 283 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: audit 2023-12-07T17:07:59.549545+0000 mon.a (mon.0) 284 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:00.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: audit 2023-12-07T17:07:59.550202+0000 mon.a (mon.0) 285 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:00.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: audit 2023-12-07T17:08:00.543564+0000 mon.a (mon.0) 286 : audit 1 from='osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:00.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: cluster 2023-12-07T17:08:00.547459+0000 mon.a (mon.0) 287 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2023-12-07T17:08:00.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:00 smithi064 bash[36207]: audit 2023-12-07T17:08:00.548462+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:01.223 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2023-12-07T17:08:01.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:01 smithi184 bash[31336]: cluster 2023-12-07T17:08:00.319815+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:01.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:01 smithi184 bash[31336]: audit 2023-12-07T17:08:00.557596+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:01.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:01 smithi184 bash[31336]: audit 2023-12-07T17:08:01.176603+0000 mon.a (mon.0) 290 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:01 smithi184 bash[31336]: audit 2023-12-07T17:08:01.182170+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:01 smithi184 bash[31336]: audit 2023-12-07T17:08:01.218270+0000 mon.a (mon.0) 292 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:01.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[25056]: cluster 2023-12-07T17:08:00.319815+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[25056]: audit 2023-12-07T17:08:00.557596+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[25056]: audit 2023-12-07T17:08:01.176603+0000 mon.a (mon.0) 290 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[25056]: audit 2023-12-07T17:08:01.182170+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[25056]: audit 2023-12-07T17:08:01.218270+0000 mon.a (mon.0) 292 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[36207]: cluster 2023-12-07T17:08:00.319815+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[36207]: audit 2023-12-07T17:08:00.557596+0000 mon.a (mon.0) 289 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[36207]: audit 2023-12-07T17:08:01.176603+0000 mon.a (mon.0) 290 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[36207]: audit 2023-12-07T17:08:01.182170+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:01 smithi064 bash[36207]: audit 2023-12-07T17:08:01.218270+0000 mon.a (mon.0) 292 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:02.022 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.0.service 2023-12-07T17:08:02.024 INFO:tasks.cephadm:Deploying osd.1 on smithi064 with /dev/nvme3n1... 2023-12-07T17:08:02.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme3n1 2023-12-07T17:08:02.262 INFO:journalctl@ceph.osd.0.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:08:02.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: cluster 2023-12-07T17:07:59.780469+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:02.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: cluster 2023-12-07T17:07:59.780604+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:02.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: audit 2023-12-07T17:08:01.552961+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:02.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: cluster 2023-12-07T17:08:01.565640+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702] boot 2023-12-07T17:08:02.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: cluster 2023-12-07T17:08:01.565698+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2023-12-07T17:08:02.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:02 smithi184 bash[31336]: audit 2023-12-07T17:08:01.565913+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:02.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: cluster 2023-12-07T17:07:59.780469+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:02.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: cluster 2023-12-07T17:07:59.780604+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: audit 2023-12-07T17:08:01.552961+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: cluster 2023-12-07T17:08:01.565640+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702] boot 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: cluster 2023-12-07T17:08:01.565698+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[25056]: audit 2023-12-07T17:08:01.565913+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: cluster 2023-12-07T17:07:59.780469+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: cluster 2023-12-07T17:07:59.780604+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: audit 2023-12-07T17:08:01.552961+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: cluster 2023-12-07T17:08:01.565640+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.64:6802/4255677702,v1:172.21.15.64:6803/4255677702] boot 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: cluster 2023-12-07T17:08:01.565698+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2023-12-07T17:08:02.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:02 smithi064 bash[36207]: audit 2023-12-07T17:08:01.565913+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:08:03.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:03 smithi064 bash[25056]: cluster 2023-12-07T17:08:02.320141+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:03.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:03 smithi064 bash[25056]: cluster 2023-12-07T17:08:02.570507+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2023-12-07T17:08:03.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:03 smithi064 bash[36207]: cluster 2023-12-07T17:08:02.320141+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:03.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:03 smithi064 bash[36207]: cluster 2023-12-07T17:08:02.570507+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2023-12-07T17:08:04.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:03 smithi184 bash[31336]: cluster 2023-12-07T17:08:02.320141+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T17:08:04.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:03 smithi184 bash[31336]: cluster 2023-12-07T17:08:02.570507+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2023-12-07T17:08:05.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:05 smithi064 bash[25056]: cluster 2023-12-07T17:08:04.320570+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:05.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:05 smithi064 bash[36207]: cluster 2023-12-07T17:08:04.320570+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:06.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:05 smithi184 bash[31336]: cluster 2023-12-07T17:08:04.320570+0000 mgr.y (mgr.14152) 112 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:06.177 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:08:07.595 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:07 smithi064 bash[25056]: cluster 2023-12-07T17:08:06.321015+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:07.596 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:07 smithi064 bash[36207]: cluster 2023-12-07T17:08:06.321015+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:08.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:07 smithi184 bash[31336]: cluster 2023-12-07T17:08:06.321015+0000 mgr.y (mgr.14152) 113 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:09.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:09 smithi064 bash[36207]: cluster 2023-12-07T17:08:08.321430+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:09.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:09 smithi064 bash[25056]: cluster 2023-12-07T17:08:08.321430+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:10.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:09 smithi184 bash[31336]: cluster 2023-12-07T17:08:08.321430+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/nvme3n1 2023-12-07T17:08:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:08:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2023-12-07T17:08:10.681 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-12-07T17:08:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-12-07T17:08:10.682 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148561 s, 706 MB/s 2023-12-07T17:08:10.682 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-12-07T17:08:11.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi064:/dev/nvme3n1 2023-12-07T17:08:11.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:11 smithi064 bash[36207]: cluster 2023-12-07T17:08:10.321900+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:11.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:11 smithi064 bash[25056]: cluster 2023-12-07T17:08:10.321900+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:12.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:11 smithi184 bash[31336]: cluster 2023-12-07T17:08:10.321900+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: cluster 2023-12-07T17:08:12.322369+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.275862+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.282672+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.284158+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.289597+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.292336+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.293604+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[25056]: audit 2023-12-07T17:08:13.300450+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: cluster 2023-12-07T17:08:12.322369+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.275862+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.282672+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.284158+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.289597+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.292336+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.293604+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:13 smithi064 bash[36207]: audit 2023-12-07T17:08:13.300450+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: cluster 2023-12-07T17:08:12.322369+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.275862+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.282672+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.284158+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.289597+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.292336+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.293604+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:13 smithi184 bash[31336]: audit 2023-12-07T17:08:13.300450+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:14.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:14 smithi064 bash[25056]: cephadm 2023-12-07T17:08:13.265391+0000 mgr.y (mgr.14152) 117 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:08:14.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:14 smithi064 bash[25056]: cephadm 2023-12-07T17:08:13.285166+0000 mgr.y (mgr.14152) 118 : cephadm 1 Adjusting osd_memory_target on smithi064 to 16241M 2023-12-07T17:08:14.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:14 smithi064 bash[36207]: cephadm 2023-12-07T17:08:13.265391+0000 mgr.y (mgr.14152) 117 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:08:14.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:14 smithi064 bash[36207]: cephadm 2023-12-07T17:08:13.285166+0000 mgr.y (mgr.14152) 118 : cephadm 1 Adjusting osd_memory_target on smithi064 to 16241M 2023-12-07T17:08:15.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:14 smithi184 bash[31336]: cephadm 2023-12-07T17:08:13.265391+0000 mgr.y (mgr.14152) 117 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:08:15.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:14 smithi184 bash[31336]: cephadm 2023-12-07T17:08:13.285166+0000 mgr.y (mgr.14152) 118 : cephadm 1 Adjusting osd_memory_target on smithi064 to 16241M 2023-12-07T17:08:15.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:15 smithi064 bash[25056]: cluster 2023-12-07T17:08:14.322851+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:15.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:15 smithi064 bash[36207]: cluster 2023-12-07T17:08:14.322851+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:16.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:15 smithi184 bash[31336]: cluster 2023-12-07T17:08:14.322851+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:16.704 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:08:17.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:17 smithi064 bash[25056]: cluster 2023-12-07T17:08:16.323259+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:17.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:17 smithi064 bash[36207]: cluster 2023-12-07T17:08:16.323259+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:18.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:17 smithi184 bash[31336]: cluster 2023-12-07T17:08:16.323259+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:20.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:19 smithi184 bash[31336]: cluster 2023-12-07T17:08:18.323800+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:20.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:19 smithi064 bash[36207]: cluster 2023-12-07T17:08:18.323800+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:20.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:19 smithi064 bash[25056]: cluster 2023-12-07T17:08:18.323800+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:21.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:20 smithi184 bash[31336]: audit 2023-12-07T17:08:19.683602+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:08:21.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:20 smithi184 bash[31336]: audit 2023-12-07T17:08:19.689425+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:08:21.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:20 smithi184 bash[31336]: audit 2023-12-07T17:08:19.690719+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:21.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[36207]: audit 2023-12-07T17:08:19.683602+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:08:21.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[36207]: audit 2023-12-07T17:08:19.689425+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:08:21.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[36207]: audit 2023-12-07T17:08:19.690719+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:21.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[25056]: audit 2023-12-07T17:08:19.683602+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:08:21.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[25056]: audit 2023-12-07T17:08:19.689425+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:08:21.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:20 smithi064 bash[25056]: audit 2023-12-07T17:08:19.690719+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:22.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:21 smithi184 bash[31336]: audit 2023-12-07T17:08:19.677135+0000 mgr.y (mgr.14152) 122 : audit 0 from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:08:22.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:21 smithi184 bash[31336]: cluster 2023-12-07T17:08:20.324190+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:22.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:21 smithi064 bash[25056]: audit 2023-12-07T17:08:19.677135+0000 mgr.y (mgr.14152) 122 : audit 0 from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:08:22.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:21 smithi064 bash[25056]: cluster 2023-12-07T17:08:20.324190+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:22.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:21 smithi064 bash[36207]: audit 2023-12-07T17:08:19.677135+0000 mgr.y (mgr.14152) 122 : audit 0 from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:08:22.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:21 smithi064 bash[36207]: cluster 2023-12-07T17:08:20.324190+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:23.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:22 smithi184 bash[31336]: cluster 2023-12-07T17:08:22.324674+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:23.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:22 smithi064 bash[25056]: cluster 2023-12-07T17:08:22.324674+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:23.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:22 smithi064 bash[36207]: cluster 2023-12-07T17:08:22.324674+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:25.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:25 smithi064 bash[25056]: cluster 2023-12-07T17:08:24.325176+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:25.717 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:25 smithi064 bash[36207]: cluster 2023-12-07T17:08:24.325176+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:25.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:25 smithi184 bash[31336]: cluster 2023-12-07T17:08:24.325176+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:27.615 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:27 smithi064 bash[25056]: cluster 2023-12-07T17:08:26.325483+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:27.616 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:27 smithi064 bash[36207]: cluster 2023-12-07T17:08:26.325483+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:27.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:27 smithi184 bash[31336]: cluster 2023-12-07T17:08:26.325483+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:28.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:28 smithi184 bash[31336]: audit 2023-12-07T17:08:28.422696+0000 mon.c (mon.2) 4 : audit 1 from='client.? 172.21.15.64:0/3292891525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:28 smithi184 bash[31336]: audit 2023-12-07T17:08:28.423143+0000 mon.a (mon.0) 308 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:28 smithi184 bash[31336]: audit 2023-12-07T17:08:28.425481+0000 mon.a (mon.0) 309 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]': finished 2023-12-07T17:08:28.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:28 smithi184 bash[31336]: cluster 2023-12-07T17:08:28.430717+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2023-12-07T17:08:28.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:28 smithi184 bash[31336]: audit 2023-12-07T17:08:28.431251+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:28.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[25056]: audit 2023-12-07T17:08:28.422696+0000 mon.c (mon.2) 4 : audit 1 from='client.? 172.21.15.64:0/3292891525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[25056]: audit 2023-12-07T17:08:28.423143+0000 mon.a (mon.0) 308 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[25056]: audit 2023-12-07T17:08:28.425481+0000 mon.a (mon.0) 309 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]': finished 2023-12-07T17:08:28.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[25056]: cluster 2023-12-07T17:08:28.430717+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2023-12-07T17:08:28.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[25056]: audit 2023-12-07T17:08:28.431251+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:28.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[36207]: audit 2023-12-07T17:08:28.422696+0000 mon.c (mon.2) 4 : audit 1 from='client.? 172.21.15.64:0/3292891525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[36207]: audit 2023-12-07T17:08:28.423143+0000 mon.a (mon.0) 308 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]: dispatch 2023-12-07T17:08:28.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[36207]: audit 2023-12-07T17:08:28.425481+0000 mon.a (mon.0) 309 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fda1ac8-8149-447f-ab1f-5c6192928f19"}]': finished 2023-12-07T17:08:28.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[36207]: cluster 2023-12-07T17:08:28.430717+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2023-12-07T17:08:28.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:28 smithi064 bash[36207]: audit 2023-12-07T17:08:28.431251+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:29.813 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:29 smithi184 bash[31336]: cluster 2023-12-07T17:08:28.325947+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:29.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:29 smithi064 bash[25056]: cluster 2023-12-07T17:08:28.325947+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:29.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:29 smithi064 bash[36207]: cluster 2023-12-07T17:08:28.325947+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:30.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:30 smithi184 bash[31336]: audit 2023-12-07T17:08:30.110779+0000 mon.c (mon.2) 5 : audit 0 from='client.? 172.21.15.64:0/549422011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:08:30.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:30 smithi064 bash[25056]: audit 2023-12-07T17:08:30.110779+0000 mon.c (mon.2) 5 : audit 0 from='client.? 172.21.15.64:0/549422011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:08:30.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:30 smithi064 bash[36207]: audit 2023-12-07T17:08:30.110779+0000 mon.c (mon.2) 5 : audit 0 from='client.? 172.21.15.64:0/549422011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:08:31.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:31 smithi184 bash[31336]: cluster 2023-12-07T17:08:30.326455+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:31.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:31 smithi064 bash[36207]: cluster 2023-12-07T17:08:30.326455+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:31.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:31 smithi064 bash[25056]: cluster 2023-12-07T17:08:30.326455+0000 mgr.y (mgr.14152) 128 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:33.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:33 smithi064 bash[25056]: cluster 2023-12-07T17:08:32.326954+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:33.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:33 smithi064 bash[36207]: cluster 2023-12-07T17:08:32.326954+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:34.063 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:33 smithi184 bash[31336]: cluster 2023-12-07T17:08:32.326954+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:35.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:35 smithi064 bash[25056]: cluster 2023-12-07T17:08:34.327446+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:35.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:35 smithi064 bash[36207]: cluster 2023-12-07T17:08:34.327446+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:36.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:35 smithi184 bash[31336]: cluster 2023-12-07T17:08:34.327446+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:37.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:37 smithi064 bash[25056]: cluster 2023-12-07T17:08:36.328017+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:37.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:37 smithi064 bash[36207]: cluster 2023-12-07T17:08:36.328017+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:38.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:37 smithi184 bash[31336]: cluster 2023-12-07T17:08:36.328017+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:39.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:39 smithi064 bash[25056]: cluster 2023-12-07T17:08:38.328463+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:39.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:39 smithi064 bash[36207]: cluster 2023-12-07T17:08:38.328463+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:40.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:39 smithi184 bash[31336]: cluster 2023-12-07T17:08:38.328463+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:41.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:41 smithi064 bash[25056]: cluster 2023-12-07T17:08:40.328779+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:41.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:41 smithi064 bash[36207]: cluster 2023-12-07T17:08:40.328779+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:42.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:41 smithi184 bash[31336]: cluster 2023-12-07T17:08:40.328779+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:42.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:42 smithi064 bash[25056]: audit 2023-12-07T17:08:42.489574+0000 mon.a (mon.0) 312 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T17:08:42.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:42 smithi064 bash[25056]: audit 2023-12-07T17:08:42.491335+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:42.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:42 smithi064 bash[36207]: audit 2023-12-07T17:08:42.489574+0000 mon.a (mon.0) 312 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T17:08:42.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:42 smithi064 bash[36207]: audit 2023-12-07T17:08:42.491335+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:43.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:42 smithi184 bash[31336]: audit 2023-12-07T17:08:42.489574+0000 mon.a (mon.0) 312 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T17:08:43.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:42 smithi184 bash[31336]: audit 2023-12-07T17:08:42.491335+0000 mon.a (mon.0) 313 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:43.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:43 smithi064 bash[25056]: cluster 2023-12-07T17:08:42.329215+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:43.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:43 smithi064 bash[25056]: cephadm 2023-12-07T17:08:42.492695+0000 mgr.y (mgr.14152) 135 : cephadm 1 Deploying daemon osd.1 on smithi064 2023-12-07T17:08:43.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:43 smithi064 bash[36207]: cluster 2023-12-07T17:08:42.329215+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:43.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:43 smithi064 bash[36207]: cephadm 2023-12-07T17:08:42.492695+0000 mgr.y (mgr.14152) 135 : cephadm 1 Deploying daemon osd.1 on smithi064 2023-12-07T17:08:44.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:43 smithi184 bash[31336]: cluster 2023-12-07T17:08:42.329215+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:44.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:43 smithi184 bash[31336]: cephadm 2023-12-07T17:08:42.492695+0000 mgr.y (mgr.14152) 135 : cephadm 1 Deploying daemon osd.1 on smithi064 2023-12-07T17:08:45.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:45 smithi064 bash[36207]: cluster 2023-12-07T17:08:44.329598+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:45.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:45 smithi064 bash[25056]: cluster 2023-12-07T17:08:44.329598+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:46.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:45 smithi184 bash[31336]: cluster 2023-12-07T17:08:44.329598+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[25056]: cluster 2023-12-07T17:08:46.330031+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[25056]: audit 2023-12-07T17:08:47.453227+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[25056]: audit 2023-12-07T17:08:47.463042+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[25056]: audit 2023-12-07T17:08:47.472764+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[36207]: cluster 2023-12-07T17:08:46.330031+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[36207]: audit 2023-12-07T17:08:47.453227+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[36207]: audit 2023-12-07T17:08:47.463042+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:47 smithi064 bash[36207]: audit 2023-12-07T17:08:47.472764+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:48.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:47 smithi184 bash[31336]: cluster 2023-12-07T17:08:46.330031+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:48.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:47 smithi184 bash[31336]: audit 2023-12-07T17:08:47.453227+0000 mon.a (mon.0) 314 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:48.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:47 smithi184 bash[31336]: audit 2023-12-07T17:08:47.463042+0000 mon.a (mon.0) 315 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:48.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:47 smithi184 bash[31336]: audit 2023-12-07T17:08:47.472764+0000 mon.a (mon.0) 316 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:49.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:49 smithi064 bash[25056]: cluster 2023-12-07T17:08:48.330373+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:49.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:49 smithi064 bash[36207]: cluster 2023-12-07T17:08:48.330373+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:50.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:49 smithi184 bash[31336]: cluster 2023-12-07T17:08:48.330373+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v101: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:51.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:51 smithi064 bash[25056]: cluster 2023-12-07T17:08:50.330826+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:51.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:51 smithi064 bash[36207]: cluster 2023-12-07T17:08:50.330826+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:52.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:51 smithi184 bash[31336]: cluster 2023-12-07T17:08:50.330826+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:54.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:53 smithi184 bash[31336]: cluster 2023-12-07T17:08:52.331302+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:54.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:53 smithi064 bash[25056]: cluster 2023-12-07T17:08:52.331302+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:54.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:53 smithi064 bash[36207]: cluster 2023-12-07T17:08:52.331302+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:55.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.114168+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.121061+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.122286+0000 mon.a (mon.0) 319 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.123264+0000 mon.a (mon.0) 320 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.128169+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: cluster 2023-12-07T17:08:54.331856+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:55.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[36207]: audit 2023-12-07T17:08:54.664238+0000 mon.a (mon.0) 322 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.114168+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.121061+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.122286+0000 mon.a (mon.0) 319 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.123264+0000 mon.a (mon.0) 320 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.128169+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: cluster 2023-12-07T17:08:54.331856+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:55 smithi064 bash[25056]: audit 2023-12-07T17:08:54.664238+0000 mon.a (mon.0) 322 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.114168+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.121061+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.122286+0000 mon.a (mon.0) 319 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.123264+0000 mon.a (mon.0) 320 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.128169+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: cluster 2023-12-07T17:08:54.331856+0000 mgr.y (mgr.14152) 141 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:55 smithi184 bash[31336]: audit 2023-12-07T17:08:54.664238+0000 mon.a (mon.0) 322 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T17:08:56.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[25056]: audit 2023-12-07T17:08:55.130797+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[25056]: cluster 2023-12-07T17:08:55.134358+0000 mon.a (mon.0) 324 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[25056]: audit 2023-12-07T17:08:55.134791+0000 mon.a (mon.0) 325 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[25056]: audit 2023-12-07T17:08:55.135077+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[36207]: audit 2023-12-07T17:08:55.130797+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[36207]: cluster 2023-12-07T17:08:55.134358+0000 mon.a (mon.0) 324 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[36207]: audit 2023-12-07T17:08:55.134791+0000 mon.a (mon.0) 325 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:56.502 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:56 smithi064 bash[36207]: audit 2023-12-07T17:08:55.135077+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:56.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:56 smithi184 bash[31336]: audit 2023-12-07T17:08:55.130797+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T17:08:56.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:56 smithi184 bash[31336]: cluster 2023-12-07T17:08:55.134358+0000 mon.a (mon.0) 324 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2023-12-07T17:08:56.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:56 smithi184 bash[31336]: audit 2023-12-07T17:08:55.134791+0000 mon.a (mon.0) 325 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:08:56.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:56 smithi184 bash[31336]: audit 2023-12-07T17:08:55.135077+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.404 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[36207]: audit 2023-12-07T17:08:56.133328+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:57.404 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[36207]: cluster 2023-12-07T17:08:56.138831+0000 mon.a (mon.0) 328 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[36207]: audit 2023-12-07T17:08:56.140057+0000 mon.a (mon.0) 329 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[36207]: audit 2023-12-07T17:08:56.147098+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[36207]: cluster 2023-12-07T17:08:56.332229+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[25056]: audit 2023-12-07T17:08:56.133328+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[25056]: cluster 2023-12-07T17:08:56.138831+0000 mon.a (mon.0) 328 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2023-12-07T17:08:57.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[25056]: audit 2023-12-07T17:08:56.140057+0000 mon.a (mon.0) 329 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[25056]: audit 2023-12-07T17:08:56.147098+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:57 smithi064 bash[25056]: cluster 2023-12-07T17:08:56.332229+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:57.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:57 smithi184 bash[31336]: audit 2023-12-07T17:08:56.133328+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:08:57.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:57 smithi184 bash[31336]: cluster 2023-12-07T17:08:56.138831+0000 mon.a (mon.0) 328 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2023-12-07T17:08:57.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:57 smithi184 bash[31336]: audit 2023-12-07T17:08:56.140057+0000 mon.a (mon.0) 329 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:57 smithi184 bash[31336]: audit 2023-12-07T17:08:56.147098+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:57.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:57 smithi184 bash[31336]: cluster 2023-12-07T17:08:56.332229+0000 mgr.y (mgr.14152) 142 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T17:08:58.202 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 1 on host 'smithi064' 2023-12-07T17:08:58.490 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: cluster 2023-12-07T17:08:55.703564+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:58.490 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: cluster 2023-12-07T17:08:55.703701+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:58.490 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: audit 2023-12-07T17:08:57.147284+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:58.490 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: cluster 2023-12-07T17:08:57.158093+0000 mon.a (mon.0) 332 : cluster 1 osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528] boot 2023-12-07T17:08:58.490 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: cluster 2023-12-07T17:08:57.158152+0000 mon.a (mon.0) 333 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2023-12-07T17:08:58.491 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[36207]: audit 2023-12-07T17:08:57.158700+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:58.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: cluster 2023-12-07T17:08:55.703564+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:58.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: cluster 2023-12-07T17:08:55.703701+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:58.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: audit 2023-12-07T17:08:57.147284+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:58.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: cluster 2023-12-07T17:08:57.158093+0000 mon.a (mon.0) 332 : cluster 1 osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528] boot 2023-12-07T17:08:58.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: cluster 2023-12-07T17:08:57.158152+0000 mon.a (mon.0) 333 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2023-12-07T17:08:58.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:58 smithi064 bash[25056]: audit 2023-12-07T17:08:57.158700+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: cluster 2023-12-07T17:08:55.703564+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: cluster 2023-12-07T17:08:55.703701+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: audit 2023-12-07T17:08:57.147284+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: cluster 2023-12-07T17:08:57.158093+0000 mon.a (mon.0) 332 : cluster 1 osd.1 [v2:172.21.15.64:6810/1129839528,v1:172.21.15.64:6811/1129839528] boot 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: cluster 2023-12-07T17:08:57.158152+0000 mon.a (mon.0) 333 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2023-12-07T17:08:58.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:58 smithi184 bash[31336]: audit 2023-12-07T17:08:57.158700+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:08:59.046 DEBUG:teuthology.orchestra.run.smithi064:osd.1> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.1.service 2023-12-07T17:08:59.049 INFO:tasks.cephadm:Deploying osd.2 on smithi064 with /dev/nvme2n1... 2023-12-07T17:08:59.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme2n1 2023-12-07T17:08:59.169 INFO:journalctl@ceph.osd.1.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:08:59.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[25056]: cluster 2023-12-07T17:08:58.164618+0000 mon.a (mon.0) 335 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2023-12-07T17:08:59.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[25056]: audit 2023-12-07T17:08:58.183212+0000 mon.a (mon.0) 336 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[25056]: audit 2023-12-07T17:08:58.190981+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[25056]: audit 2023-12-07T17:08:58.221842+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:59.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[25056]: cluster 2023-12-07T17:08:58.332509+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:08:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[36207]: cluster 2023-12-07T17:08:58.164618+0000 mon.a (mon.0) 335 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2023-12-07T17:08:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[36207]: audit 2023-12-07T17:08:58.183212+0000 mon.a (mon.0) 336 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[36207]: audit 2023-12-07T17:08:58.190981+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[36207]: audit 2023-12-07T17:08:58.221842+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:08:59 smithi064 bash[36207]: cluster 2023-12-07T17:08:58.332509+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:08:59.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:59 smithi184 bash[31336]: cluster 2023-12-07T17:08:58.164618+0000 mon.a (mon.0) 335 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2023-12-07T17:08:59.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:59 smithi184 bash[31336]: audit 2023-12-07T17:08:58.183212+0000 mon.a (mon.0) 336 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:59 smithi184 bash[31336]: audit 2023-12-07T17:08:58.190981+0000 mon.a (mon.0) 337 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:08:59.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:59 smithi184 bash[31336]: audit 2023-12-07T17:08:58.221842+0000 mon.a (mon.0) 338 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:08:59.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:08:59 smithi184 bash[31336]: cluster 2023-12-07T17:08:58.332509+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:01.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:01 smithi184 bash[31336]: cluster 2023-12-07T17:09:00.333060+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:01.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:01 smithi064 bash[25056]: cluster 2023-12-07T17:09:00.333060+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:01.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:01 smithi064 bash[36207]: cluster 2023-12-07T17:09:00.333060+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:09:03.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:03 smithi184 bash[31336]: cluster 2023-12-07T17:09:02.333684+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:03.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:03 smithi064 bash[25056]: cluster 2023-12-07T17:09:02.333684+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:03.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:03 smithi064 bash[36207]: cluster 2023-12-07T17:09:02.333684+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:05.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:05 smithi184 bash[31336]: cluster 2023-12-07T17:09:04.334357+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:05.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:05 smithi064 bash[25056]: cluster 2023-12-07T17:09:04.334357+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:05.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:05 smithi064 bash[36207]: cluster 2023-12-07T17:09:04.334357+0000 mgr.y (mgr.14152) 146 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:07.847 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:07 smithi064 bash[25056]: cluster 2023-12-07T17:09:06.334678+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:07.847 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:07 smithi064 bash[36207]: cluster 2023-12-07T17:09:06.334678+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:08.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:07 smithi184 bash[31336]: cluster 2023-12-07T17:09:06.334678+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/nvme2n1 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0167421 s, 626 MB/s 2023-12-07T17:09:08.561 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-12-07T17:09:09.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi064:/dev/nvme2n1 2023-12-07T17:09:09.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:09 smithi064 bash[36207]: cluster 2023-12-07T17:09:08.335125+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:09.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:09 smithi064 bash[25056]: cluster 2023-12-07T17:09:08.335125+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:10.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:09 smithi184 bash[31336]: cluster 2023-12-07T17:09:08.335125+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v115: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:11.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:11 smithi064 bash[36207]: cluster 2023-12-07T17:09:10.335547+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:11.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:11 smithi064 bash[25056]: cluster 2023-12-07T17:09:10.335547+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:12.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:11 smithi184 bash[31336]: cluster 2023-12-07T17:09:10.335547+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:12.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: cephadm 2023-12-07T17:09:11.589722+0000 mgr.y (mgr.14152) 150 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.601100+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.609108+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.610665+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.611708+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: cephadm 2023-12-07T17:09:11.612434+0000 mgr.y (mgr.14152) 151 : cephadm 1 Adjusting osd_memory_target on smithi064 to 8120M 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.616629+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.620408+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.621674+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[25056]: audit 2023-12-07T17:09:11.629860+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: cephadm 2023-12-07T17:09:11.589722+0000 mgr.y (mgr.14152) 150 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.601100+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.609108+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.610665+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.611708+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: cephadm 2023-12-07T17:09:11.612434+0000 mgr.y (mgr.14152) 151 : cephadm 1 Adjusting osd_memory_target on smithi064 to 8120M 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.616629+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.620408+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.621674+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:12.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:12 smithi064 bash[36207]: audit 2023-12-07T17:09:11.629860+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:13.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: cephadm 2023-12-07T17:09:11.589722+0000 mgr.y (mgr.14152) 150 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:09:13.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.601100+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:13.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.609108+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:13.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.610665+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.611708+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: cephadm 2023-12-07T17:09:11.612434+0000 mgr.y (mgr.14152) 151 : cephadm 1 Adjusting osd_memory_target on smithi064 to 8120M 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.616629+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.620408+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.621674+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:13.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:12 smithi184 bash[31336]: audit 2023-12-07T17:09:11.629860+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:13 smithi184 bash[31336]: cluster 2023-12-07T17:09:12.336023+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:14.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:13 smithi064 bash[25056]: cluster 2023-12-07T17:09:12.336023+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:14.116 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:13 smithi064 bash[36207]: cluster 2023-12-07T17:09:12.336023+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:14.127 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:09:15.968 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:15 smithi064 bash[36207]: cluster 2023-12-07T17:09:14.336455+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:15.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:15 smithi064 bash[25056]: cluster 2023-12-07T17:09:14.336455+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:16.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:15 smithi184 bash[31336]: cluster 2023-12-07T17:09:14.336455+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:17 smithi184 bash[31336]: cluster 2023-12-07T17:09:16.336960+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:17 smithi184 bash[31336]: audit 2023-12-07T17:09:17.421837+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:09:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:17 smithi184 bash[31336]: audit 2023-12-07T17:09:17.425136+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:09:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:17 smithi184 bash[31336]: audit 2023-12-07T17:09:17.426183+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:18.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[25056]: cluster 2023-12-07T17:09:16.336960+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[25056]: audit 2023-12-07T17:09:17.421837+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[25056]: audit 2023-12-07T17:09:17.425136+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[25056]: audit 2023-12-07T17:09:17.426183+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[36207]: cluster 2023-12-07T17:09:16.336960+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[36207]: audit 2023-12-07T17:09:17.421837+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[36207]: audit 2023-12-07T17:09:17.425136+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:09:18.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:17 smithi064 bash[36207]: audit 2023-12-07T17:09:17.426183+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:19.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:18 smithi184 bash[31336]: audit 2023-12-07T17:09:17.414450+0000 mgr.y (mgr.14152) 155 : audit 0 from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:09:19.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:18 smithi184 bash[31336]: cluster 2023-12-07T17:09:18.337466+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:19.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:18 smithi064 bash[25056]: audit 2023-12-07T17:09:17.414450+0000 mgr.y (mgr.14152) 155 : audit 0 from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:09:19.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:18 smithi064 bash[25056]: cluster 2023-12-07T17:09:18.337466+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:19.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:18 smithi064 bash[36207]: audit 2023-12-07T17:09:17.414450+0000 mgr.y (mgr.14152) 155 : audit 0 from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:09:19.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:18 smithi064 bash[36207]: cluster 2023-12-07T17:09:18.337466+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:21.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:21 smithi064 bash[25056]: cluster 2023-12-07T17:09:20.337915+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:21.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:21 smithi064 bash[36207]: cluster 2023-12-07T17:09:20.337915+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:22.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:21 smithi184 bash[31336]: cluster 2023-12-07T17:09:20.337915+0000 mgr.y (mgr.14152) 157 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:23.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:23 smithi064 bash[25056]: cluster 2023-12-07T17:09:22.338395+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:23.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:23 smithi064 bash[36207]: cluster 2023-12-07T17:09:22.338395+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:24.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:23 smithi184 bash[31336]: cluster 2023-12-07T17:09:22.338395+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:25.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:25 smithi064 bash[25056]: cluster 2023-12-07T17:09:24.338794+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:25.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:25 smithi064 bash[36207]: cluster 2023-12-07T17:09:24.338794+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:26.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:25 smithi184 bash[31336]: cluster 2023-12-07T17:09:24.338794+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:26.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[25056]: audit 2023-12-07T17:09:26.253692+0000 mon.a (mon.0) 350 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]: dispatch 2023-12-07T17:09:26.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[25056]: audit 2023-12-07T17:09:26.256640+0000 mon.a (mon.0) 351 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]': finished 2023-12-07T17:09:26.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[25056]: cluster 2023-12-07T17:09:26.265055+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2023-12-07T17:09:26.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[25056]: audit 2023-12-07T17:09:26.265528+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:26.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[36207]: audit 2023-12-07T17:09:26.253692+0000 mon.a (mon.0) 350 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]: dispatch 2023-12-07T17:09:26.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[36207]: audit 2023-12-07T17:09:26.256640+0000 mon.a (mon.0) 351 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]': finished 2023-12-07T17:09:26.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[36207]: cluster 2023-12-07T17:09:26.265055+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2023-12-07T17:09:26.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:26 smithi064 bash[36207]: audit 2023-12-07T17:09:26.265528+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:27.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:26 smithi184 bash[31336]: audit 2023-12-07T17:09:26.253692+0000 mon.a (mon.0) 350 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]: dispatch 2023-12-07T17:09:27.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:26 smithi184 bash[31336]: audit 2023-12-07T17:09:26.256640+0000 mon.a (mon.0) 351 : audit 1 from='client.? 172.21.15.64:0/3964325326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36cb5cff-a8e8-46cb-86c5-55243883da50"}]': finished 2023-12-07T17:09:27.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:26 smithi184 bash[31336]: cluster 2023-12-07T17:09:26.265055+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2023-12-07T17:09:27.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:26 smithi184 bash[31336]: audit 2023-12-07T17:09:26.265528+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:27.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:27 smithi064 bash[36207]: cluster 2023-12-07T17:09:26.339141+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:27.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:27 smithi064 bash[25056]: cluster 2023-12-07T17:09:26.339141+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:28.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:27 smithi184 bash[31336]: cluster 2023-12-07T17:09:26.339141+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:28.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:28 smithi064 bash[25056]: audit 2023-12-07T17:09:28.046266+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.64:0/3358029940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:09:28.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:28 smithi064 bash[36207]: audit 2023-12-07T17:09:28.046266+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.64:0/3358029940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:09:29.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:28 smithi184 bash[31336]: audit 2023-12-07T17:09:28.046266+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.64:0/3358029940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:09:29.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:29 smithi064 bash[25056]: cluster 2023-12-07T17:09:28.339630+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:29.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:29 smithi064 bash[36207]: cluster 2023-12-07T17:09:28.339630+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:30.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:29 smithi184 bash[31336]: cluster 2023-12-07T17:09:28.339630+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:31.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:31 smithi064 bash[25056]: cluster 2023-12-07T17:09:30.340102+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:31.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:31 smithi064 bash[36207]: cluster 2023-12-07T17:09:30.340102+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:32.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:31 smithi184 bash[31336]: cluster 2023-12-07T17:09:30.340102+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:33.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:33 smithi064 bash[25056]: cluster 2023-12-07T17:09:32.340520+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v128: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:33.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:33 smithi064 bash[36207]: cluster 2023-12-07T17:09:32.340520+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v128: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:34.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:33 smithi184 bash[31336]: cluster 2023-12-07T17:09:32.340520+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v128: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:35.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:35 smithi064 bash[36207]: cluster 2023-12-07T17:09:34.341029+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:35.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:35 smithi064 bash[25056]: cluster 2023-12-07T17:09:34.341029+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:36.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:35 smithi184 bash[31336]: cluster 2023-12-07T17:09:34.341029+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v129: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:37.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:37 smithi064 bash[25056]: cluster 2023-12-07T17:09:36.341452+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:37.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:37 smithi064 bash[36207]: cluster 2023-12-07T17:09:36.341452+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:38.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:37 smithi184 bash[31336]: cluster 2023-12-07T17:09:36.341452+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:39.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:39 smithi064 bash[36207]: cluster 2023-12-07T17:09:38.341951+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v131: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:39.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:39 smithi064 bash[25056]: cluster 2023-12-07T17:09:38.341951+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v131: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:40.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:39 smithi184 bash[31336]: cluster 2023-12-07T17:09:38.341951+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v131: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:40.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:40 smithi064 bash[25056]: audit 2023-12-07T17:09:40.498021+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T17:09:40.969 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:40 smithi064 bash[25056]: audit 2023-12-07T17:09:40.499487+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:40.969 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:40 smithi064 bash[36207]: audit 2023-12-07T17:09:40.498021+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T17:09:40.969 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:40 smithi064 bash[36207]: audit 2023-12-07T17:09:40.499487+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:41.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:40 smithi184 bash[31336]: audit 2023-12-07T17:09:40.498021+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T17:09:41.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:40 smithi184 bash[31336]: audit 2023-12-07T17:09:40.499487+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:41.991 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:41 smithi064 bash[25056]: cluster 2023-12-07T17:09:40.342363+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:41.991 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:41 smithi064 bash[25056]: cephadm 2023-12-07T17:09:40.500621+0000 mgr.y (mgr.14152) 168 : cephadm 1 Deploying daemon osd.2 on smithi064 2023-12-07T17:09:41.991 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:41 smithi064 bash[36207]: cluster 2023-12-07T17:09:40.342363+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:41.991 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:41 smithi064 bash[36207]: cephadm 2023-12-07T17:09:40.500621+0000 mgr.y (mgr.14152) 168 : cephadm 1 Deploying daemon osd.2 on smithi064 2023-12-07T17:09:42.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:41 smithi184 bash[31336]: cluster 2023-12-07T17:09:40.342363+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:42.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:41 smithi184 bash[31336]: cephadm 2023-12-07T17:09:40.500621+0000 mgr.y (mgr.14152) 168 : cephadm 1 Deploying daemon osd.2 on smithi064 2023-12-07T17:09:43.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:43 smithi064 bash[25056]: cluster 2023-12-07T17:09:42.342905+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v133: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:43.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:43 smithi064 bash[36207]: cluster 2023-12-07T17:09:42.342905+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v133: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:44.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:43 smithi184 bash[31336]: cluster 2023-12-07T17:09:42.342905+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v133: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:46.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:45 smithi184 bash[31336]: cluster 2023-12-07T17:09:44.343551+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v134: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:46.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:45 smithi184 bash[31336]: audit 2023-12-07T17:09:45.340990+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:45 smithi184 bash[31336]: audit 2023-12-07T17:09:45.349216+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:45 smithi184 bash[31336]: audit 2023-12-07T17:09:45.350054+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[25056]: cluster 2023-12-07T17:09:44.343551+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v134: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[25056]: audit 2023-12-07T17:09:45.340990+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[25056]: audit 2023-12-07T17:09:45.349216+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[25056]: audit 2023-12-07T17:09:45.350054+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[36207]: cluster 2023-12-07T17:09:44.343551+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v134: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[36207]: audit 2023-12-07T17:09:45.340990+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[36207]: audit 2023-12-07T17:09:45.349216+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:45 smithi064 bash[36207]: audit 2023-12-07T17:09:45.350054+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:47.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:47 smithi064 bash[25056]: cluster 2023-12-07T17:09:46.344056+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v135: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:47 smithi064 bash[36207]: cluster 2023-12-07T17:09:46.344056+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v135: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:48.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:47 smithi184 bash[31336]: cluster 2023-12-07T17:09:46.344056+0000 mgr.y (mgr.14152) 171 : cluster 0 pgmap v135: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:48.953 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:48 smithi064 bash[25056]: cluster 2023-12-07T17:09:48.344527+0000 mgr.y (mgr.14152) 172 : cluster 0 pgmap v136: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:48.954 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:48 smithi064 bash[36207]: cluster 2023-12-07T17:09:48.344527+0000 mgr.y (mgr.14152) 172 : cluster 0 pgmap v136: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:49.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:48 smithi184 bash[31336]: cluster 2023-12-07T17:09:48.344527+0000 mgr.y (mgr.14152) 172 : cluster 0 pgmap v136: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:51.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:51 smithi064 bash[25056]: cluster 2023-12-07T17:09:50.345017+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v137: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:51.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:51 smithi064 bash[36207]: cluster 2023-12-07T17:09:50.345017+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v137: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:52.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:51 smithi184 bash[31336]: cluster 2023-12-07T17:09:50.345017+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v137: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:53.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:51.947082+0000 mon.a (mon.0) 360 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:51.954295+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:51.955633+0000 mon.a (mon.0) 362 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:51.956955+0000 mon.a (mon.0) 363 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:51.963116+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: cluster 2023-12-07T17:09:52.345437+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v138: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:52 smithi184 bash[31336]: audit 2023-12-07T17:09:52.613258+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T17:09:53.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:51.947082+0000 mon.a (mon.0) 360 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:51.954295+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:51.955633+0000 mon.a (mon.0) 362 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:51.956955+0000 mon.a (mon.0) 363 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:51.963116+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: cluster 2023-12-07T17:09:52.345437+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v138: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[25056]: audit 2023-12-07T17:09:52.613258+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:51.947082+0000 mon.a (mon.0) 360 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:51.954295+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:51.955633+0000 mon.a (mon.0) 362 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:09:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:51.956955+0000 mon.a (mon.0) 363 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:09:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:51.963116+0000 mon.a (mon.0) 364 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: cluster 2023-12-07T17:09:52.345437+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v138: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:52 smithi064 bash[36207]: audit 2023-12-07T17:09:52.613258+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T17:09:54.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:53 smithi184 bash[31336]: audit 2023-12-07T17:09:52.968341+0000 mon.a (mon.0) 366 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T17:09:54.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:53 smithi184 bash[31336]: cluster 2023-12-07T17:09:52.972694+0000 mon.a (mon.0) 367 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2023-12-07T17:09:54.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:53 smithi184 bash[31336]: audit 2023-12-07T17:09:52.973234+0000 mon.a (mon.0) 368 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:09:54.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:53 smithi184 bash[31336]: audit 2023-12-07T17:09:52.973797+0000 mon.a (mon.0) 369 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:54.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[25056]: audit 2023-12-07T17:09:52.968341+0000 mon.a (mon.0) 366 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T17:09:54.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[25056]: cluster 2023-12-07T17:09:52.972694+0000 mon.a (mon.0) 367 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[25056]: audit 2023-12-07T17:09:52.973234+0000 mon.a (mon.0) 368 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[25056]: audit 2023-12-07T17:09:52.973797+0000 mon.a (mon.0) 369 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[36207]: audit 2023-12-07T17:09:52.968341+0000 mon.a (mon.0) 366 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[36207]: cluster 2023-12-07T17:09:52.972694+0000 mon.a (mon.0) 367 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[36207]: audit 2023-12-07T17:09:52.973234+0000 mon.a (mon.0) 368 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:09:54.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:53 smithi064 bash[36207]: audit 2023-12-07T17:09:52.973797+0000 mon.a (mon.0) 369 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:54 smithi184 bash[31336]: audit 2023-12-07T17:09:53.970904+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:09:55.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:54 smithi184 bash[31336]: cluster 2023-12-07T17:09:53.977557+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2023-12-07T17:09:55.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:54 smithi184 bash[31336]: audit 2023-12-07T17:09:53.978914+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:54 smithi184 bash[31336]: audit 2023-12-07T17:09:53.985496+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:54 smithi184 bash[31336]: cluster 2023-12-07T17:09:54.345925+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v141: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[25056]: audit 2023-12-07T17:09:53.970904+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[25056]: cluster 2023-12-07T17:09:53.977557+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[25056]: audit 2023-12-07T17:09:53.978914+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[25056]: audit 2023-12-07T17:09:53.985496+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[25056]: cluster 2023-12-07T17:09:54.345925+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v141: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[36207]: audit 2023-12-07T17:09:53.970904+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:09:55.338 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[36207]: cluster 2023-12-07T17:09:53.977557+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2023-12-07T17:09:55.339 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[36207]: audit 2023-12-07T17:09:53.978914+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.339 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[36207]: audit 2023-12-07T17:09:53.985496+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:55.339 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:54 smithi064 bash[36207]: cluster 2023-12-07T17:09:54.345925+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v141: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T17:09:56.080 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 2 on host 'smithi064' 2023-12-07T17:09:56.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: cluster 2023-12-07T17:09:53.615080+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:09:56.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: cluster 2023-12-07T17:09:53.615209+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:09:56.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: audit 2023-12-07T17:09:54.986500+0000 mon.a (mon.0) 374 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: cluster 2023-12-07T17:09:54.993774+0000 mon.a (mon.0) 375 : cluster 1 osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840] boot 2023-12-07T17:09:56.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: cluster 2023-12-07T17:09:54.993818+0000 mon.a (mon.0) 376 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2023-12-07T17:09:56.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:55 smithi184 bash[31336]: audit 2023-12-07T17:09:54.994158+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: cluster 2023-12-07T17:09:53.615080+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: cluster 2023-12-07T17:09:53.615209+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: audit 2023-12-07T17:09:54.986500+0000 mon.a (mon.0) 374 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: cluster 2023-12-07T17:09:54.993774+0000 mon.a (mon.0) 375 : cluster 1 osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840] boot 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: cluster 2023-12-07T17:09:54.993818+0000 mon.a (mon.0) 376 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[25056]: audit 2023-12-07T17:09:54.994158+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: cluster 2023-12-07T17:09:53.615080+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: cluster 2023-12-07T17:09:53.615209+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: audit 2023-12-07T17:09:54.986500+0000 mon.a (mon.0) 374 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: cluster 2023-12-07T17:09:54.993774+0000 mon.a (mon.0) 375 : cluster 1 osd.2 [v2:172.21.15.64:6818/135846840,v1:172.21.15.64:6819/135846840] boot 2023-12-07T17:09:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: cluster 2023-12-07T17:09:54.993818+0000 mon.a (mon.0) 376 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2023-12-07T17:09:56.333 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:55 smithi064 bash[36207]: audit 2023-12-07T17:09:54.994158+0000 mon.a (mon.0) 377 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:09:56.851 DEBUG:teuthology.orchestra.run.smithi064:osd.2> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.2.service 2023-12-07T17:09:56.854 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/nvme1n1... 2023-12-07T17:09:56.854 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme1n1 2023-12-07T17:09:56.965 INFO:journalctl@ceph.osd.2.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.034046+0000 mon.a (mon.0) 378 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.041812+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.055418+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: cluster 2023-12-07T17:09:56.346251+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.499617+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.940386+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: cluster 2023-12-07T17:09:56.944275+0000 mon.a (mon.0) 383 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2023-12-07T17:09:57.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[36207]: audit 2023-12-07T17:09:56.945195+0000 mon.a (mon.0) 384 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.034046+0000 mon.a (mon.0) 378 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.041812+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.055418+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: cluster 2023-12-07T17:09:56.346251+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.499617+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.940386+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: cluster 2023-12-07T17:09:56.944275+0000 mon.a (mon.0) 383 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2023-12-07T17:09:57.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:57 smithi064 bash[25056]: audit 2023-12-07T17:09:56.945195+0000 mon.a (mon.0) 384 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:57.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.034046+0000 mon.a (mon.0) 378 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.041812+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.055418+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: cluster 2023-12-07T17:09:56.346251+0000 mgr.y (mgr.14152) 176 : cluster 0 pgmap v143: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.499617+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.940386+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: cluster 2023-12-07T17:09:56.944275+0000 mon.a (mon.0) 383 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2023-12-07T17:09:57.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:57 smithi184 bash[31336]: audit 2023-12-07T17:09:56.945195+0000 mon.a (mon.0) 384 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T17:09:59.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:58 smithi184 bash[31336]: audit 2023-12-07T17:09:57.942738+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:59.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:58 smithi184 bash[31336]: cluster 2023-12-07T17:09:57.949268+0000 mon.a (mon.0) 386 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2023-12-07T17:09:59.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:58 smithi184 bash[31336]: cluster 2023-12-07T17:09:58.346623+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v146: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[25056]: audit 2023-12-07T17:09:57.942738+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[25056]: cluster 2023-12-07T17:09:57.949268+0000 mon.a (mon.0) 386 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[25056]: cluster 2023-12-07T17:09:58.346623+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v146: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[36207]: audit 2023-12-07T17:09:57.942738+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[36207]: cluster 2023-12-07T17:09:57.949268+0000 mon.a (mon.0) 386 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2023-12-07T17:09:59.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:58 smithi064 bash[36207]: cluster 2023-12-07T17:09:58.346623+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v146: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:00.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:59 smithi184 bash[31336]: cluster 2023-12-07T17:09:58.963569+0000 mon.a (mon.0) 387 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2023-12-07T17:10:00.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:59 smithi184 bash[31336]: cluster 2023-12-07T17:09:58.963609+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2023-12-07T17:10:00.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:09:59 smithi184 bash[31336]: audit 2023-12-07T17:09:59.392074+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:00.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[25056]: cluster 2023-12-07T17:09:58.963569+0000 mon.a (mon.0) 387 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2023-12-07T17:10:00.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[25056]: cluster 2023-12-07T17:09:58.963609+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2023-12-07T17:10:00.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[25056]: audit 2023-12-07T17:09:59.392074+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:00.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[36207]: cluster 2023-12-07T17:09:58.963569+0000 mon.a (mon.0) 387 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2023-12-07T17:10:00.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[36207]: cluster 2023-12-07T17:09:58.963609+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2023-12-07T17:10:00.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:09:59 smithi064 bash[36207]: audit 2023-12-07T17:09:59.392074+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:01.036 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:10:01.314 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:00 smithi184 bash[31336]: cluster 2023-12-07T17:10:00.000150+0000 mon.a (mon.0) 390 : cluster 1 overall HEALTH_OK 2023-12-07T17:10:01.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:00 smithi184 bash[31336]: cluster 2023-12-07T17:10:00.346935+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v148: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:01.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:00 smithi184 bash[31336]: audit 2023-12-07T17:10:00.356502+0000 mon.a (mon.0) 391 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:01.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[36207]: cluster 2023-12-07T17:10:00.000150+0000 mon.a (mon.0) 390 : cluster 1 overall HEALTH_OK 2023-12-07T17:10:01.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[36207]: cluster 2023-12-07T17:10:00.346935+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v148: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:01.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[36207]: audit 2023-12-07T17:10:00.356502+0000 mon.a (mon.0) 391 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:01.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[25056]: cluster 2023-12-07T17:10:00.000150+0000 mon.a (mon.0) 390 : cluster 1 overall HEALTH_OK 2023-12-07T17:10:01.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[25056]: cluster 2023-12-07T17:10:00.346935+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v148: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:01.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:00 smithi064 bash[25056]: audit 2023-12-07T17:10:00.356502+0000 mon.a (mon.0) 391 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:01.331095+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:01.332467+0000 mon.a (mon.0) 393 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.246646+0000 mon.a (mon.0) 394 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.247497+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.248057+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.248550+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.276512+0000 mon.a (mon.0) 398 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.278962+0000 mon.a (mon.0) 399 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.279170+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:03 smithi184 bash[31336]: audit 2023-12-07T17:10:02.279340+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:01.331095+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:01.332467+0000 mon.a (mon.0) 393 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.246646+0000 mon.a (mon.0) 394 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.247497+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.248057+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.248550+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.276512+0000 mon.a (mon.0) 398 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.278962+0000 mon.a (mon.0) 399 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.279170+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[36207]: audit 2023-12-07T17:10:02.279340+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:01.331095+0000 mon.a (mon.0) 392 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:01.332467+0000 mon.a (mon.0) 393 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.246646+0000 mon.a (mon.0) 394 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.247497+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.248057+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.248550+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.276512+0000 mon.a (mon.0) 398 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.278962+0000 mon.a (mon.0) 399 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.279170+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:03.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:03 smithi064 bash[25056]: audit 2023-12-07T17:10:02.279340+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:02.278712+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: cluster 2023-12-07T17:10:02.347239+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v149: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.151103+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.184362+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.186049+0000 mon.c (mon.2) 6 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.186305+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.186574+0000 mon.a (mon.0) 404 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:04.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:03.186806+0000 mon.a (mon.0) 405 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[25056]: audit 2023-12-07T17:10:04.117115+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:02.278712+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: cluster 2023-12-07T17:10:02.347239+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v149: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.151103+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.184362+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.186049+0000 mon.c (mon.2) 6 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.186305+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.186574+0000 mon.a (mon.0) 404 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:03.186806+0000 mon.a (mon.0) 405 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:04.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:04 smithi064 bash[36207]: audit 2023-12-07T17:10:04.117115+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:02.278712+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: cluster 2023-12-07T17:10:02.347239+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v149: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.151103+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.184362+0000 mon.a (mon.0) 402 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.186049+0000 mon.c (mon.2) 6 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.186305+0000 mon.a (mon.0) 403 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.186574+0000 mon.a (mon.0) 404 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:03.186806+0000 mon.a (mon.0) 405 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:10:04.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:04 smithi184 bash[31336]: audit 2023-12-07T17:10:04.117115+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T17:10:05.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:05 smithi184 bash[31336]: cluster 2023-12-07T17:10:04.347660+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:05.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:05 smithi064 bash[25056]: cluster 2023-12-07T17:10:04.347660+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:05.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:05 smithi064 bash[36207]: cluster 2023-12-07T17:10:04.347660+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:05.976 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/nvme1n1 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0153651 s, 682 MB/s 2023-12-07T17:10:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-12-07T17:10:06.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi064:/dev/nvme1n1 2023-12-07T17:10:07.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:07 smithi064 bash[25056]: cluster 2023-12-07T17:10:06.348133+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:07.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:07 smithi064 bash[36207]: cluster 2023-12-07T17:10:06.348133+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:08.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:07 smithi184 bash[31336]: cluster 2023-12-07T17:10:06.348133+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: cluster 2023-12-07T17:10:08.348663+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-12-07T17:10:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.221008+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.229735+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.231551+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.232933+0000 mon.a (mon.0) 409 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.234165+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.239996+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.243598+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.244931+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[25056]: audit 2023-12-07T17:10:09.251591+0000 mon.a (mon.0) 414 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: cluster 2023-12-07T17:10:08.348663+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.221008+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.229735+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.231551+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.232933+0000 mon.a (mon.0) 409 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.234165+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:09.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.239996+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:09.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.243598+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:09.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.244931+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:09.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:09 smithi064 bash[36207]: audit 2023-12-07T17:10:09.251591+0000 mon.a (mon.0) 414 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: cluster 2023-12-07T17:10:08.348663+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-12-07T17:10:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.221008+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.229735+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.231551+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.232933+0000 mon.a (mon.0) 409 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.234165+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:10:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.239996+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.243598+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.244931+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:09 smithi184 bash[31336]: audit 2023-12-07T17:10:09.251591+0000 mon.a (mon.0) 414 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:10.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:10 smithi064 bash[36207]: cephadm 2023-12-07T17:10:09.210695+0000 mgr.y (mgr.14152) 183 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:10:10.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:10 smithi064 bash[36207]: cephadm 2023-12-07T17:10:09.235136+0000 mgr.y (mgr.14152) 184 : cephadm 1 Adjusting osd_memory_target on smithi064 to 5413M 2023-12-07T17:10:10.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:10 smithi064 bash[25056]: cephadm 2023-12-07T17:10:09.210695+0000 mgr.y (mgr.14152) 183 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:10:10.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:10 smithi064 bash[25056]: cephadm 2023-12-07T17:10:09.235136+0000 mgr.y (mgr.14152) 184 : cephadm 1 Adjusting osd_memory_target on smithi064 to 5413M 2023-12-07T17:10:11.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:10 smithi184 bash[31336]: cephadm 2023-12-07T17:10:09.210695+0000 mgr.y (mgr.14152) 183 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:10:11.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:10 smithi184 bash[31336]: cephadm 2023-12-07T17:10:09.235136+0000 mgr.y (mgr.14152) 184 : cephadm 1 Adjusting osd_memory_target on smithi064 to 5413M 2023-12-07T17:10:11.457 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:10:11.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:11 smithi064 bash[36207]: cluster 2023-12-07T17:10:10.349224+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-07T17:10:11.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:11 smithi064 bash[25056]: cluster 2023-12-07T17:10:10.349224+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-07T17:10:12.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:11 smithi184 bash[31336]: cluster 2023-12-07T17:10:10.349224+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-07T17:10:13.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:13 smithi064 bash[36207]: cluster 2023-12-07T17:10:12.349644+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-12-07T17:10:13.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:13 smithi064 bash[25056]: cluster 2023-12-07T17:10:12.349644+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-12-07T17:10:14.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:13 smithi184 bash[31336]: cluster 2023-12-07T17:10:12.349644+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[36207]: audit 2023-12-07T17:10:14.488265+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[36207]: audit 2023-12-07T17:10:14.491995+0000 mon.a (mon.0) 416 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[36207]: audit 2023-12-07T17:10:14.493517+0000 mon.a (mon.0) 417 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[25056]: audit 2023-12-07T17:10:14.488265+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[25056]: audit 2023-12-07T17:10:14.491995+0000 mon.a (mon.0) 416 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:10:14.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:14 smithi064 bash[25056]: audit 2023-12-07T17:10:14.493517+0000 mon.a (mon.0) 417 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:15.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:14 smithi184 bash[31336]: audit 2023-12-07T17:10:14.488265+0000 mon.a (mon.0) 415 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:10:15.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:14 smithi184 bash[31336]: audit 2023-12-07T17:10:14.491995+0000 mon.a (mon.0) 416 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:10:15.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:14 smithi184 bash[31336]: audit 2023-12-07T17:10:14.493517+0000 mon.a (mon.0) 417 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:15.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:15 smithi064 bash[25056]: cluster 2023-12-07T17:10:14.350164+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-12-07T17:10:15.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:15 smithi064 bash[25056]: audit 2023-12-07T17:10:14.485223+0000 mgr.y (mgr.14152) 188 : audit 0 from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:10:15.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:15 smithi064 bash[36207]: cluster 2023-12-07T17:10:14.350164+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-12-07T17:10:15.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:15 smithi064 bash[36207]: audit 2023-12-07T17:10:14.485223+0000 mgr.y (mgr.14152) 188 : audit 0 from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:10:16.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:15 smithi184 bash[31336]: cluster 2023-12-07T17:10:14.350164+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-12-07T17:10:16.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:15 smithi184 bash[31336]: audit 2023-12-07T17:10:14.485223+0000 mgr.y (mgr.14152) 188 : audit 0 from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:10:18.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:17 smithi184 bash[31336]: cluster 2023-12-07T17:10:16.350852+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:18.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:17 smithi064 bash[36207]: cluster 2023-12-07T17:10:16.350852+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:18.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:17 smithi064 bash[25056]: cluster 2023-12-07T17:10:16.350852+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:19.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:19 smithi064 bash[25056]: cluster 2023-12-07T17:10:18.351515+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:19.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:19 smithi064 bash[36207]: cluster 2023-12-07T17:10:18.351515+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:20.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:19 smithi184 bash[31336]: cluster 2023-12-07T17:10:18.351515+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-12-07T17:10:21.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:21 smithi064 bash[25056]: cluster 2023-12-07T17:10:20.352067+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:21.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:21 smithi064 bash[36207]: cluster 2023-12-07T17:10:20.352067+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:22.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:21 smithi184 bash[31336]: cluster 2023-12-07T17:10:20.352067+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:24.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: cluster 2023-12-07T17:10:22.352577+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: audit 2023-12-07T17:10:23.357234+0000 mon.c (mon.2) 8 : audit 1 from='client.? 172.21.15.64:0/1215841614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: audit 2023-12-07T17:10:23.357997+0000 mon.a (mon.0) 418 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: audit 2023-12-07T17:10:23.361392+0000 mon.a (mon.0) 419 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]': finished 2023-12-07T17:10:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: cluster 2023-12-07T17:10:23.366760+0000 mon.a (mon.0) 420 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2023-12-07T17:10:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:23 smithi184 bash[31336]: audit 2023-12-07T17:10:23.367320+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:24.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: cluster 2023-12-07T17:10:22.352577+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:24.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: audit 2023-12-07T17:10:23.357234+0000 mon.c (mon.2) 8 : audit 1 from='client.? 172.21.15.64:0/1215841614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: audit 2023-12-07T17:10:23.357997+0000 mon.a (mon.0) 418 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: audit 2023-12-07T17:10:23.361392+0000 mon.a (mon.0) 419 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]': finished 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: cluster 2023-12-07T17:10:23.366760+0000 mon.a (mon.0) 420 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[25056]: audit 2023-12-07T17:10:23.367320+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: cluster 2023-12-07T17:10:22.352577+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: audit 2023-12-07T17:10:23.357234+0000 mon.c (mon.2) 8 : audit 1 from='client.? 172.21.15.64:0/1215841614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: audit 2023-12-07T17:10:23.357997+0000 mon.a (mon.0) 418 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]: dispatch 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: audit 2023-12-07T17:10:23.361392+0000 mon.a (mon.0) 419 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22c06021-9405-4ee7-8b82-1b3ee1a7b5bf"}]': finished 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: cluster 2023-12-07T17:10:23.366760+0000 mon.a (mon.0) 420 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2023-12-07T17:10:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:23 smithi064 bash[36207]: audit 2023-12-07T17:10:23.367320+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:26.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:25 smithi184 bash[31336]: cluster 2023-12-07T17:10:24.353143+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:26.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:25 smithi184 bash[31336]: audit 2023-12-07T17:10:25.066419+0000 mon.a (mon.0) 422 : audit 0 from='client.? 172.21.15.64:0/1243645951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:10:26.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:25 smithi064 bash[25056]: cluster 2023-12-07T17:10:24.353143+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:26.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:25 smithi064 bash[25056]: audit 2023-12-07T17:10:25.066419+0000 mon.a (mon.0) 422 : audit 0 from='client.? 172.21.15.64:0/1243645951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:10:26.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:25 smithi064 bash[36207]: cluster 2023-12-07T17:10:24.353143+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:26.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:25 smithi064 bash[36207]: audit 2023-12-07T17:10:25.066419+0000 mon.a (mon.0) 422 : audit 0 from='client.? 172.21.15.64:0/1243645951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:10:28.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:27 smithi184 bash[31336]: cluster 2023-12-07T17:10:26.353682+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:28.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:27 smithi064 bash[25056]: cluster 2023-12-07T17:10:26.353682+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:28.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:27 smithi064 bash[36207]: cluster 2023-12-07T17:10:26.353682+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:30.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:29 smithi184 bash[31336]: cluster 2023-12-07T17:10:28.354367+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:30.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:29 smithi064 bash[25056]: cluster 2023-12-07T17:10:28.354367+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:30.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:29 smithi064 bash[36207]: cluster 2023-12-07T17:10:28.354367+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:32.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:31 smithi184 bash[31336]: cluster 2023-12-07T17:10:30.354935+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:32.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:31 smithi064 bash[36207]: cluster 2023-12-07T17:10:30.354935+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:32.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:31 smithi064 bash[25056]: cluster 2023-12-07T17:10:30.354935+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:33.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:32 smithi184 bash[31336]: cluster 2023-12-07T17:10:32.355499+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:33.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:32 smithi064 bash[25056]: cluster 2023-12-07T17:10:32.355499+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:33.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:32 smithi064 bash[36207]: cluster 2023-12-07T17:10:32.355499+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:35.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:35 smithi064 bash[36207]: cluster 2023-12-07T17:10:34.356136+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:35.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:35 smithi064 bash[25056]: cluster 2023-12-07T17:10:34.356136+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:35.814 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:35 smithi184 bash[31336]: cluster 2023-12-07T17:10:34.356136+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[25056]: cluster 2023-12-07T17:10:36.356644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[25056]: audit 2023-12-07T17:10:37.336135+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[25056]: audit 2023-12-07T17:10:37.337510+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[36207]: cluster 2023-12-07T17:10:36.356644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[36207]: audit 2023-12-07T17:10:37.336135+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T17:10:37.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:37 smithi064 bash[36207]: audit 2023-12-07T17:10:37.337510+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:38.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:37 smithi184 bash[31336]: cluster 2023-12-07T17:10:36.356644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:38.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:37 smithi184 bash[31336]: audit 2023-12-07T17:10:37.336135+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T17:10:38.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:37 smithi184 bash[31336]: audit 2023-12-07T17:10:37.337510+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:38.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:38 smithi064 bash[25056]: cephadm 2023-12-07T17:10:37.338972+0000 mgr.y (mgr.14152) 200 : cephadm 1 Deploying daemon osd.3 on smithi064 2023-12-07T17:10:38.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:38 smithi064 bash[36207]: cephadm 2023-12-07T17:10:37.338972+0000 mgr.y (mgr.14152) 200 : cephadm 1 Deploying daemon osd.3 on smithi064 2023-12-07T17:10:39.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:38 smithi184 bash[31336]: cephadm 2023-12-07T17:10:37.338972+0000 mgr.y (mgr.14152) 200 : cephadm 1 Deploying daemon osd.3 on smithi064 2023-12-07T17:10:39.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:39 smithi064 bash[25056]: cluster 2023-12-07T17:10:38.357199+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:39.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:39 smithi064 bash[36207]: cluster 2023-12-07T17:10:38.357199+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:40.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:39 smithi184 bash[31336]: cluster 2023-12-07T17:10:38.357199+0000 mgr.y (mgr.14152) 201 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:41.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:41 smithi064 bash[36207]: cluster 2023-12-07T17:10:40.357711+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:41.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:41 smithi064 bash[25056]: cluster 2023-12-07T17:10:40.357711+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:42.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:41 smithi184 bash[31336]: cluster 2023-12-07T17:10:40.357711+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:43.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[25056]: audit 2023-12-07T17:10:42.322621+0000 mon.a (mon.0) 425 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[25056]: audit 2023-12-07T17:10:42.323115+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:43.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[25056]: audit 2023-12-07T17:10:42.330092+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[25056]: cluster 2023-12-07T17:10:42.358064+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:43.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[36207]: audit 2023-12-07T17:10:42.322621+0000 mon.a (mon.0) 425 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.618 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[36207]: audit 2023-12-07T17:10:42.323115+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:43.618 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[36207]: audit 2023-12-07T17:10:42.330092+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.618 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:43 smithi064 bash[36207]: cluster 2023-12-07T17:10:42.358064+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:43 smithi184 bash[31336]: audit 2023-12-07T17:10:42.322621+0000 mon.a (mon.0) 425 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:43 smithi184 bash[31336]: audit 2023-12-07T17:10:42.323115+0000 mon.a (mon.0) 426 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:43 smithi184 bash[31336]: audit 2023-12-07T17:10:42.330092+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:43 smithi184 bash[31336]: cluster 2023-12-07T17:10:42.358064+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:45.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:45 smithi064 bash[25056]: cluster 2023-12-07T17:10:44.358597+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:45.709 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:45 smithi064 bash[36207]: cluster 2023-12-07T17:10:44.358597+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:45.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:45 smithi184 bash[31336]: cluster 2023-12-07T17:10:44.358597+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:47.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:47 smithi064 bash[25056]: cluster 2023-12-07T17:10:46.359195+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:47.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:47 smithi064 bash[36207]: cluster 2023-12-07T17:10:46.359195+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:48.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:47 smithi184 bash[31336]: cluster 2023-12-07T17:10:46.359195+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: audit 2023-12-07T17:10:47.915435+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: audit 2023-12-07T17:10:47.923081+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: cluster 2023-12-07T17:10:48.359839+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: audit 2023-12-07T17:10:48.547422+0000 mon.a (mon.0) 430 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: audit 2023-12-07T17:10:48.548771+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:49.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[25056]: audit 2023-12-07T17:10:48.555308+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: audit 2023-12-07T17:10:47.915435+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: audit 2023-12-07T17:10:47.923081+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: cluster 2023-12-07T17:10:48.359839+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: audit 2023-12-07T17:10:48.547422+0000 mon.a (mon.0) 430 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: audit 2023-12-07T17:10:48.548771+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:49.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:48 smithi064 bash[36207]: audit 2023-12-07T17:10:48.555308+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: audit 2023-12-07T17:10:47.915435+0000 mon.a (mon.0) 428 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: audit 2023-12-07T17:10:47.923081+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: cluster 2023-12-07T17:10:48.359839+0000 mgr.y (mgr.14152) 206 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: audit 2023-12-07T17:10:48.547422+0000 mon.a (mon.0) 430 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: audit 2023-12-07T17:10:48.548771+0000 mon.a (mon.0) 431 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:10:49.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:48 smithi184 bash[31336]: audit 2023-12-07T17:10:48.555308+0000 mon.a (mon.0) 432 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:50.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:49 smithi184 bash[31336]: audit 2023-12-07T17:10:49.421360+0000 mon.c (mon.2) 9 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:50.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:49 smithi184 bash[31336]: audit 2023-12-07T17:10:49.421905+0000 mon.a (mon.0) 433 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:50.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:49 smithi064 bash[36207]: audit 2023-12-07T17:10:49.421360+0000 mon.c (mon.2) 9 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:50.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:49 smithi064 bash[36207]: audit 2023-12-07T17:10:49.421905+0000 mon.a (mon.0) 433 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:50.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:49 smithi064 bash[25056]: audit 2023-12-07T17:10:49.421360+0000 mon.c (mon.2) 9 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:50.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:49 smithi064 bash[25056]: audit 2023-12-07T17:10:49.421905+0000 mon.a (mon.0) 433 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: audit 2023-12-07T17:10:49.931049+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: cluster 2023-12-07T17:10:49.933865+0000 mon.a (mon.0) 435 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: audit 2023-12-07T17:10:49.934198+0000 mon.a (mon.0) 436 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: audit 2023-12-07T17:10:49.935844+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: audit 2023-12-07T17:10:49.936281+0000 mon.a (mon.0) 437 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:50 smithi184 bash[31336]: cluster 2023-12-07T17:10:50.360352+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: audit 2023-12-07T17:10:49.931049+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: cluster 2023-12-07T17:10:49.933865+0000 mon.a (mon.0) 435 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: audit 2023-12-07T17:10:49.934198+0000 mon.a (mon.0) 436 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: audit 2023-12-07T17:10:49.935844+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: audit 2023-12-07T17:10:49.936281+0000 mon.a (mon.0) 437 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.346 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[36207]: cluster 2023-12-07T17:10:50.360352+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: audit 2023-12-07T17:10:49.931049+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: cluster 2023-12-07T17:10:49.933865+0000 mon.a (mon.0) 435 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: audit 2023-12-07T17:10:49.934198+0000 mon.a (mon.0) 436 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: audit 2023-12-07T17:10:49.935844+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: audit 2023-12-07T17:10:49.936281+0000 mon.a (mon.0) 437 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-12-07T17:10:51.347 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:50 smithi064 bash[25056]: cluster 2023-12-07T17:10:50.360352+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[25056]: audit 2023-12-07T17:10:50.933727+0000 mon.a (mon.0) 438 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[25056]: cluster 2023-12-07T17:10:50.938895+0000 mon.a (mon.0) 439 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[25056]: audit 2023-12-07T17:10:50.939819+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[25056]: audit 2023-12-07T17:10:50.946212+0000 mon.a (mon.0) 441 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[36207]: audit 2023-12-07T17:10:50.933727+0000 mon.a (mon.0) 438 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[36207]: cluster 2023-12-07T17:10:50.938895+0000 mon.a (mon.0) 439 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2023-12-07T17:10:52.262 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[36207]: audit 2023-12-07T17:10:50.939819+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:52.263 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:51 smithi064 bash[36207]: audit 2023-12-07T17:10:50.946212+0000 mon.a (mon.0) 441 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:52.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:51 smithi184 bash[31336]: audit 2023-12-07T17:10:50.933727+0000 mon.a (mon.0) 438 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-12-07T17:10:52.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:51 smithi184 bash[31336]: cluster 2023-12-07T17:10:50.938895+0000 mon.a (mon.0) 439 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2023-12-07T17:10:52.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:51 smithi184 bash[31336]: audit 2023-12-07T17:10:50.939819+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:52.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:51 smithi184 bash[31336]: audit 2023-12-07T17:10:50.946212+0000 mon.a (mon.0) 441 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:53.023 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.3.service 2023-12-07T17:10:53.025 INFO:tasks.cephadm:Deploying osd.4 on smithi184 with /dev/nvme4n1... 2023-12-07T17:10:53.026 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2023-12-07T17:10:53.207 INFO:journalctl@ceph.osd.3.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: cluster 2023-12-07T17:10:50.468024+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: cluster 2023-12-07T17:10:50.468111+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: cluster 2023-12-07T17:10:51.948752+0000 mon.a (mon.0) 442 : cluster 1 osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191] boot 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: cluster 2023-12-07T17:10:51.948822+0000 mon.a (mon.0) 443 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: audit 2023-12-07T17:10:51.949437+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: audit 2023-12-07T17:10:52.165367+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: audit 2023-12-07T17:10:52.172925+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.315 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: audit 2023-12-07T17:10:52.197839+0000 mon.a (mon.0) 447 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:53.316 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:52 smithi184 bash[31336]: cluster 2023-12-07T17:10:52.360734+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: cluster 2023-12-07T17:10:50.468024+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: cluster 2023-12-07T17:10:50.468111+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: cluster 2023-12-07T17:10:51.948752+0000 mon.a (mon.0) 442 : cluster 1 osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191] boot 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: cluster 2023-12-07T17:10:51.948822+0000 mon.a (mon.0) 443 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: audit 2023-12-07T17:10:51.949437+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: audit 2023-12-07T17:10:52.165367+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: audit 2023-12-07T17:10:52.172925+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: audit 2023-12-07T17:10:52.197839+0000 mon.a (mon.0) 447 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:53.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[25056]: cluster 2023-12-07T17:10:52.360734+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: cluster 2023-12-07T17:10:50.468024+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: cluster 2023-12-07T17:10:50.468111+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: cluster 2023-12-07T17:10:51.948752+0000 mon.a (mon.0) 442 : cluster 1 osd.3 [v2:172.21.15.64:6826/2613043191,v1:172.21.15.64:6827/2613043191] boot 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: cluster 2023-12-07T17:10:51.948822+0000 mon.a (mon.0) 443 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: audit 2023-12-07T17:10:51.949437+0000 mon.a (mon.0) 444 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: audit 2023-12-07T17:10:52.165367+0000 mon.a (mon.0) 445 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: audit 2023-12-07T17:10:52.172925+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: audit 2023-12-07T17:10:52.197839+0000 mon.a (mon.0) 447 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:10:53.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:52 smithi064 bash[36207]: cluster 2023-12-07T17:10:52.360734+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-07T17:10:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:54 smithi184 bash[31336]: cluster 2023-12-07T17:10:53.182876+0000 mon.a (mon.0) 448 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2023-12-07T17:10:54.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:54 smithi064 bash[25056]: cluster 2023-12-07T17:10:53.182876+0000 mon.a (mon.0) 448 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2023-12-07T17:10:54.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:54 smithi064 bash[36207]: cluster 2023-12-07T17:10:53.182876+0000 mon.a (mon.0) 448 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2023-12-07T17:10:55.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:55 smithi184 bash[31336]: cluster 2023-12-07T17:10:54.361414+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:55.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:55 smithi064 bash[25056]: cluster 2023-12-07T17:10:54.361414+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:55.617 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:55 smithi064 bash[36207]: cluster 2023-12-07T17:10:54.361414+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:57.753 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:10:57.862 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:57 smithi184 bash[31336]: cluster 2023-12-07T17:10:56.362033+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:57.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:57 smithi064 bash[25056]: cluster 2023-12-07T17:10:56.362033+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:57.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:57 smithi064 bash[36207]: cluster 2023-12-07T17:10:56.362033+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:59.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:10:59 smithi184 bash[31336]: cluster 2023-12-07T17:10:58.362605+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:59.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:10:59 smithi064 bash[25056]: cluster 2023-12-07T17:10:58.362605+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:10:59.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:10:59 smithi064 bash[36207]: cluster 2023-12-07T17:10:58.362605+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:00.806 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/nvme4n1 2023-12-07T17:11:00.806 INFO:teuthology.orchestra.run.smithi184.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:11:00.807 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2023-12-07T17:11:00.807 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-12-07T17:11:00.807 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-12-07T17:11:00.807 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0177769 s, 590 MB/s 2023-12-07T17:11:00.807 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-12-07T17:11:01.436 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi184:/dev/nvme4n1 2023-12-07T17:11:01.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:01 smithi184 bash[31336]: cluster 2023-12-07T17:11:00.363130+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:01.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:01 smithi064 bash[25056]: cluster 2023-12-07T17:11:00.363130+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:01.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:01 smithi064 bash[36207]: cluster 2023-12-07T17:11:00.363130+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:03.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:03 smithi064 bash[25056]: cluster 2023-12-07T17:11:02.363775+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:03.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:03 smithi064 bash[36207]: cluster 2023-12-07T17:11:02.363775+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:04.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:03 smithi184 bash[31336]: cluster 2023-12-07T17:11:02.363775+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:05.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: cephadm 2023-12-07T17:11:04.143971+0000 mgr.y (mgr.14152) 214 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:11:05.565 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.154181+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.163632+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.165556+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.166914+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.168144+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.169428+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: cephadm 2023-12-07T17:11:04.170452+0000 mgr.y (mgr.14152) 215 : cephadm 1 Adjusting osd_memory_target on smithi064 to 4060M 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.177355+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.181090+0000 mon.a (mon.0) 456 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.182318+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: audit 2023-12-07T17:11:04.188979+0000 mon.a (mon.0) 458 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:05 smithi184 bash[31336]: cluster 2023-12-07T17:11:04.364323+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: cephadm 2023-12-07T17:11:04.143971+0000 mgr.y (mgr.14152) 214 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.154181+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.163632+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.165556+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.166914+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.168144+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.169428+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: cephadm 2023-12-07T17:11:04.170452+0000 mgr.y (mgr.14152) 215 : cephadm 1 Adjusting osd_memory_target on smithi064 to 4060M 2023-12-07T17:11:05.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.177355+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.181090+0000 mon.a (mon.0) 456 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.182318+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: audit 2023-12-07T17:11:04.188979+0000 mon.a (mon.0) 458 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[25056]: cluster 2023-12-07T17:11:04.364323+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: cephadm 2023-12-07T17:11:04.143971+0000 mgr.y (mgr.14152) 214 : cephadm 1 Detected new or changed devices on smithi064 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.154181+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.163632+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.165556+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.166914+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.168144+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.169428+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:05.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: cephadm 2023-12-07T17:11:04.170452+0000 mgr.y (mgr.14152) 215 : cephadm 1 Adjusting osd_memory_target on smithi064 to 4060M 2023-12-07T17:11:05.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.177355+0000 mon.a (mon.0) 455 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.181090+0000 mon.a (mon.0) 456 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:05.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.182318+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:05.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: audit 2023-12-07T17:11:04.188979+0000 mon.a (mon.0) 458 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:05.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:05 smithi064 bash[36207]: cluster 2023-12-07T17:11:04.364323+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:06.152 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:11:07.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:07 smithi064 bash[25056]: cluster 2023-12-07T17:11:06.364872+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:07.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:07 smithi064 bash[36207]: cluster 2023-12-07T17:11:06.364872+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:08.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:07 smithi184 bash[31336]: cluster 2023-12-07T17:11:06.364872+0000 mgr.y (mgr.14152) 217 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:09.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[25056]: cluster 2023-12-07T17:11:08.365434+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[25056]: audit 2023-12-07T17:11:09.278495+0000 mon.a (mon.0) 459 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[25056]: audit 2023-12-07T17:11:09.282966+0000 mon.a (mon.0) 460 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[25056]: audit 2023-12-07T17:11:09.284527+0000 mon.a (mon.0) 461 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[36207]: cluster 2023-12-07T17:11:08.365434+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[36207]: audit 2023-12-07T17:11:09.278495+0000 mon.a (mon.0) 459 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[36207]: audit 2023-12-07T17:11:09.282966+0000 mon.a (mon.0) 460 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:11:09.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:09 smithi064 bash[36207]: audit 2023-12-07T17:11:09.284527+0000 mon.a (mon.0) 461 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:09 smithi184 bash[31336]: cluster 2023-12-07T17:11:08.365434+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:09 smithi184 bash[31336]: audit 2023-12-07T17:11:09.278495+0000 mon.a (mon.0) 459 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:11:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:09 smithi184 bash[31336]: audit 2023-12-07T17:11:09.282966+0000 mon.a (mon.0) 460 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:11:10.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:09 smithi184 bash[31336]: audit 2023-12-07T17:11:09.284527+0000 mon.a (mon.0) 461 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:10.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:10 smithi064 bash[25056]: audit 2023-12-07T17:11:09.272306+0000 mgr.y (mgr.14152) 219 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:11:10.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:10 smithi064 bash[36207]: audit 2023-12-07T17:11:09.272306+0000 mgr.y (mgr.14152) 219 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:11:11.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:10 smithi184 bash[31336]: audit 2023-12-07T17:11:09.272306+0000 mgr.y (mgr.14152) 219 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:11:11.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:11 smithi064 bash[25056]: cluster 2023-12-07T17:11:10.365964+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:11.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:11 smithi064 bash[36207]: cluster 2023-12-07T17:11:10.365964+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:12.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:11 smithi184 bash[31336]: cluster 2023-12-07T17:11:10.365964+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:14.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:13 smithi184 bash[31336]: cluster 2023-12-07T17:11:12.366515+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:14.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:13 smithi064 bash[25056]: cluster 2023-12-07T17:11:12.366515+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:14.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:13 smithi064 bash[36207]: cluster 2023-12-07T17:11:12.366515+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:15.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:15 smithi184 bash[31336]: cluster 2023-12-07T17:11:14.367097+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:16.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:15 smithi064 bash[25056]: cluster 2023-12-07T17:11:14.367097+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:16.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:15 smithi064 bash[36207]: cluster 2023-12-07T17:11:14.367097+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:18.020 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:17 smithi184 bash[31336]: cluster 2023-12-07T17:11:16.367708+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:18.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:17 smithi064 bash[25056]: cluster 2023-12-07T17:11:16.367708+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:18.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:17 smithi064 bash[36207]: cluster 2023-12-07T17:11:16.367708+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:19.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:18 smithi184 bash[31336]: audit 2023-12-07T17:11:17.809713+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.184:0/2176832717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:18 smithi184 bash[31336]: audit 2023-12-07T17:11:17.810685+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:18 smithi184 bash[31336]: audit 2023-12-07T17:11:17.814060+0000 mon.a (mon.0) 463 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]': finished 2023-12-07T17:11:19.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:18 smithi184 bash[31336]: cluster 2023-12-07T17:11:17.821505+0000 mon.a (mon.0) 464 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2023-12-07T17:11:19.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:18 smithi184 bash[31336]: audit 2023-12-07T17:11:17.821891+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:19.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[36207]: audit 2023-12-07T17:11:17.809713+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.184:0/2176832717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[36207]: audit 2023-12-07T17:11:17.810685+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[36207]: audit 2023-12-07T17:11:17.814060+0000 mon.a (mon.0) 463 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]': finished 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[36207]: cluster 2023-12-07T17:11:17.821505+0000 mon.a (mon.0) 464 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[36207]: audit 2023-12-07T17:11:17.821891+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[25056]: audit 2023-12-07T17:11:17.809713+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.184:0/2176832717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[25056]: audit 2023-12-07T17:11:17.810685+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]: dispatch 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[25056]: audit 2023-12-07T17:11:17.814060+0000 mon.a (mon.0) 463 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9be6b07a-5939-4ac2-abaa-f889afc87cdc"}]': finished 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[25056]: cluster 2023-12-07T17:11:17.821505+0000 mon.a (mon.0) 464 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2023-12-07T17:11:19.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:18 smithi064 bash[25056]: audit 2023-12-07T17:11:17.821891+0000 mon.a (mon.0) 465 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:20.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:19 smithi184 bash[31336]: cluster 2023-12-07T17:11:18.368275+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:20.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:19 smithi184 bash[31336]: audit 2023-12-07T17:11:19.414658+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.184:0/1388485320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:11:20.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:19 smithi064 bash[25056]: cluster 2023-12-07T17:11:18.368275+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:20.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:19 smithi064 bash[25056]: audit 2023-12-07T17:11:19.414658+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.184:0/1388485320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:11:20.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:19 smithi064 bash[36207]: cluster 2023-12-07T17:11:18.368275+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:20.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:19 smithi064 bash[36207]: audit 2023-12-07T17:11:19.414658+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.184:0/1388485320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:11:21.953 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:21 smithi184 bash[31336]: cluster 2023-12-07T17:11:20.368844+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:22.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:21 smithi064 bash[25056]: cluster 2023-12-07T17:11:20.368844+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:22.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:21 smithi064 bash[36207]: cluster 2023-12-07T17:11:20.368844+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:24.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:23 smithi184 bash[31336]: cluster 2023-12-07T17:11:22.369418+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:24.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:23 smithi064 bash[36207]: cluster 2023-12-07T17:11:22.369418+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:24.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:23 smithi064 bash[25056]: cluster 2023-12-07T17:11:22.369418+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:26.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:25 smithi184 bash[31336]: cluster 2023-12-07T17:11:24.369996+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:26.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:25 smithi064 bash[25056]: cluster 2023-12-07T17:11:24.369996+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:26.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:25 smithi064 bash[36207]: cluster 2023-12-07T17:11:24.369996+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:28.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:27 smithi184 bash[31336]: cluster 2023-12-07T17:11:26.370474+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:28.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:27 smithi064 bash[25056]: cluster 2023-12-07T17:11:26.370474+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:28.117 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:27 smithi064 bash[36207]: cluster 2023-12-07T17:11:26.370474+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:30.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:29 smithi184 bash[31336]: cluster 2023-12-07T17:11:28.371045+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:30.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:29 smithi064 bash[25056]: cluster 2023-12-07T17:11:28.371045+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:30.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:29 smithi064 bash[36207]: cluster 2023-12-07T17:11:28.371045+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:32.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:31 smithi184 bash[31336]: cluster 2023-12-07T17:11:30.371564+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:32.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:31 smithi184 bash[31336]: audit 2023-12-07T17:11:31.291972+0000 mon.a (mon.0) 466 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T17:11:32.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:31 smithi184 bash[31336]: audit 2023-12-07T17:11:31.293574+0000 mon.a (mon.0) 467 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:32.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[25056]: cluster 2023-12-07T17:11:30.371564+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:32.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[25056]: audit 2023-12-07T17:11:31.291972+0000 mon.a (mon.0) 466 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T17:11:32.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[25056]: audit 2023-12-07T17:11:31.293574+0000 mon.a (mon.0) 467 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:32.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[36207]: cluster 2023-12-07T17:11:30.371564+0000 mgr.y (mgr.14152) 230 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:32.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[36207]: audit 2023-12-07T17:11:31.291972+0000 mon.a (mon.0) 466 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T17:11:32.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:31 smithi064 bash[36207]: audit 2023-12-07T17:11:31.293574+0000 mon.a (mon.0) 467 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:32.860 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:32 smithi184 bash[31336]: cephadm 2023-12-07T17:11:31.294809+0000 mgr.y (mgr.14152) 231 : cephadm 1 Deploying daemon osd.4 on smithi184 2023-12-07T17:11:33.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:32 smithi064 bash[25056]: cephadm 2023-12-07T17:11:31.294809+0000 mgr.y (mgr.14152) 231 : cephadm 1 Deploying daemon osd.4 on smithi184 2023-12-07T17:11:33.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:32 smithi064 bash[36207]: cephadm 2023-12-07T17:11:31.294809+0000 mgr.y (mgr.14152) 231 : cephadm 1 Deploying daemon osd.4 on smithi184 2023-12-07T17:11:33.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:33 smithi184 bash[31336]: cluster 2023-12-07T17:11:32.372109+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:34.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:33 smithi064 bash[25056]: cluster 2023-12-07T17:11:32.372109+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:34.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:33 smithi064 bash[36207]: cluster 2023-12-07T17:11:32.372109+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:35.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:34 smithi184 bash[31336]: cluster 2023-12-07T17:11:34.372709+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:35.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:34 smithi064 bash[25056]: cluster 2023-12-07T17:11:34.372709+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:35.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:34 smithi064 bash[36207]: cluster 2023-12-07T17:11:34.372709+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:37.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:36 smithi184 bash[31336]: audit 2023-12-07T17:11:35.998821+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:36 smithi184 bash[31336]: audit 2023-12-07T17:11:36.005030+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:36 smithi184 bash[31336]: audit 2023-12-07T17:11:36.012755+0000 mon.a (mon.0) 470 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:37.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:36 smithi184 bash[31336]: cluster 2023-12-07T17:11:36.373358+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:37.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[25056]: audit 2023-12-07T17:11:35.998821+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[25056]: audit 2023-12-07T17:11:36.005030+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[25056]: audit 2023-12-07T17:11:36.012755+0000 mon.a (mon.0) 470 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[25056]: cluster 2023-12-07T17:11:36.373358+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[36207]: audit 2023-12-07T17:11:35.998821+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[36207]: audit 2023-12-07T17:11:36.005030+0000 mon.a (mon.0) 469 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[36207]: audit 2023-12-07T17:11:36.012755+0000 mon.a (mon.0) 470 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:37.368 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:36 smithi064 bash[36207]: cluster 2023-12-07T17:11:36.373358+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:39.746 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:39 smithi184 bash[31336]: cluster 2023-12-07T17:11:38.373951+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:39.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:39 smithi064 bash[25056]: cluster 2023-12-07T17:11:38.373951+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:39.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:39 smithi064 bash[36207]: cluster 2023-12-07T17:11:38.373951+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[25056]: cluster 2023-12-07T17:11:40.374609+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[25056]: audit 2023-12-07T17:11:41.355833+0000 mon.a (mon.0) 471 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[25056]: audit 2023-12-07T17:11:41.362425+0000 mon.a (mon.0) 472 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[36207]: cluster 2023-12-07T17:11:40.374609+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[36207]: audit 2023-12-07T17:11:41.355833+0000 mon.a (mon.0) 471 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:41.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:41 smithi064 bash[36207]: audit 2023-12-07T17:11:41.362425+0000 mon.a (mon.0) 472 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:41.953 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:41 smithi184 bash[31336]: cluster 2023-12-07T17:11:40.374609+0000 mgr.y (mgr.14152) 236 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:41.953 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:41 smithi184 bash[31336]: audit 2023-12-07T17:11:41.355833+0000 mon.a (mon.0) 471 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:41.954 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:41 smithi184 bash[31336]: audit 2023-12-07T17:11:41.362425+0000 mon.a (mon.0) 472 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:42.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[25056]: audit 2023-12-07T17:11:41.984978+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:42.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[25056]: audit 2023-12-07T17:11:41.986872+0000 mon.a (mon.0) 474 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:42.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[25056]: audit 2023-12-07T17:11:41.995952+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:42.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[36207]: audit 2023-12-07T17:11:41.984978+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:42.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[36207]: audit 2023-12-07T17:11:41.986872+0000 mon.a (mon.0) 474 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:42.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:42 smithi064 bash[36207]: audit 2023-12-07T17:11:41.995952+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:43.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:42 smithi184 bash[31336]: audit 2023-12-07T17:11:41.984978+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:43.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:42 smithi184 bash[31336]: audit 2023-12-07T17:11:41.986872+0000 mon.a (mon.0) 474 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:43.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:42 smithi184 bash[31336]: audit 2023-12-07T17:11:41.995952+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:43 smithi184 bash[31336]: cluster 2023-12-07T17:11:42.375201+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:43 smithi184 bash[31336]: audit 2023-12-07T17:11:42.758458+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:43 smithi184 bash[31336]: audit 2023-12-07T17:11:42.759025+0000 mon.a (mon.0) 476 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[36207]: cluster 2023-12-07T17:11:42.375201+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[36207]: audit 2023-12-07T17:11:42.758458+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[36207]: audit 2023-12-07T17:11:42.759025+0000 mon.a (mon.0) 476 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[25056]: cluster 2023-12-07T17:11:42.375201+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[25056]: audit 2023-12-07T17:11:42.758458+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:44.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:43 smithi064 bash[25056]: audit 2023-12-07T17:11:42.759025+0000 mon.a (mon.0) 476 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T17:11:44.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:43.614275+0000 mon.a (mon.0) 477 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:43.620934+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: cluster 2023-12-07T17:11:43.624299+0000 mon.a (mon.0) 478 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:43.624929+0000 mon.a (mon.0) 479 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:43.625393+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:44.617069+0000 mon.a (mon.0) 481 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: cluster 2023-12-07T17:11:44.625346+0000 mon.a (mon.0) 482 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2023-12-07T17:11:44.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:44 smithi184 bash[31336]: audit 2023-12-07T17:11:44.625827+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:44.998 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 4 on host 'smithi184' 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:43.614275+0000 mon.a (mon.0) 477 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:43.620934+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: cluster 2023-12-07T17:11:43.624299+0000 mon.a (mon.0) 478 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:43.624929+0000 mon.a (mon.0) 479 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:43.625393+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:44.617069+0000 mon.a (mon.0) 481 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: cluster 2023-12-07T17:11:44.625346+0000 mon.a (mon.0) 482 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2023-12-07T17:11:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[36207]: audit 2023-12-07T17:11:44.625827+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:43.614275+0000 mon.a (mon.0) 477 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:43.620934+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: cluster 2023-12-07T17:11:43.624299+0000 mon.a (mon.0) 478 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:43.624929+0000 mon.a (mon.0) 479 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:43.625393+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:44.617069+0000 mon.a (mon.0) 481 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: cluster 2023-12-07T17:11:44.625346+0000 mon.a (mon.0) 482 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2023-12-07T17:11:45.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:44 smithi064 bash[25056]: audit 2023-12-07T17:11:44.625827+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:45.692 DEBUG:teuthology.orchestra.run.smithi184:osd.4> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.4.service 2023-12-07T17:11:45.694 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/nvme3n1... 2023-12-07T17:11:45.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme3n1 2023-12-07T17:11:45.903 INFO:journalctl@ceph.osd.4.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:11:46.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: cluster 2023-12-07T17:11:44.375768+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: audit 2023-12-07T17:11:44.633056+0000 mon.a (mon.0) 484 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: audit 2023-12-07T17:11:44.961450+0000 mon.a (mon.0) 485 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: audit 2023-12-07T17:11:44.968003+0000 mon.a (mon.0) 486 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: audit 2023-12-07T17:11:44.988697+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: cluster 2023-12-07T17:11:45.615438+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559] boot 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: cluster 2023-12-07T17:11:45.615542+0000 mon.a (mon.0) 489 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2023-12-07T17:11:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:45 smithi184 bash[31336]: audit 2023-12-07T17:11:45.616236+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: cluster 2023-12-07T17:11:44.375768+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: audit 2023-12-07T17:11:44.633056+0000 mon.a (mon.0) 484 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: audit 2023-12-07T17:11:44.961450+0000 mon.a (mon.0) 485 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: audit 2023-12-07T17:11:44.968003+0000 mon.a (mon.0) 486 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: audit 2023-12-07T17:11:44.988697+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: cluster 2023-12-07T17:11:45.615438+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559] boot 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: cluster 2023-12-07T17:11:45.615542+0000 mon.a (mon.0) 489 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2023-12-07T17:11:46.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[36207]: audit 2023-12-07T17:11:45.616236+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: cluster 2023-12-07T17:11:44.375768+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: audit 2023-12-07T17:11:44.633056+0000 mon.a (mon.0) 484 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: audit 2023-12-07T17:11:44.961450+0000 mon.a (mon.0) 485 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: audit 2023-12-07T17:11:44.968003+0000 mon.a (mon.0) 486 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: audit 2023-12-07T17:11:44.988697+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: cluster 2023-12-07T17:11:45.615438+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.184:6800/2660335559,v1:172.21.15.184:6801/2660335559] boot 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: cluster 2023-12-07T17:11:45.615542+0000 mon.a (mon.0) 489 : cluster 0 osdmap e30: 5 total, 5 up, 5 in 2023-12-07T17:11:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:45 smithi064 bash[25056]: audit 2023-12-07T17:11:45.616236+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:11:47.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:46 smithi184 bash[31336]: cluster 2023-12-07T17:11:43.718623+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:11:47.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:46 smithi184 bash[31336]: cluster 2023-12-07T17:11:43.718714+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:11:47.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:46 smithi184 bash[31336]: cluster 2023-12-07T17:11:46.626083+0000 mon.a (mon.0) 491 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[25056]: cluster 2023-12-07T17:11:43.718623+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[25056]: cluster 2023-12-07T17:11:43.718714+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[25056]: cluster 2023-12-07T17:11:46.626083+0000 mon.a (mon.0) 491 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[36207]: cluster 2023-12-07T17:11:43.718623+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[36207]: cluster 2023-12-07T17:11:43.718714+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:11:47.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:46 smithi064 bash[36207]: cluster 2023-12-07T17:11:46.626083+0000 mon.a (mon.0) 491 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2023-12-07T17:11:48.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:47 smithi184 bash[31336]: cluster 2023-12-07T17:11:46.376271+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:48.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:47 smithi184 bash[31336]: cluster 2023-12-07T17:11:47.632667+0000 mon.a (mon.0) 492 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2023-12-07T17:11:48.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:47 smithi064 bash[25056]: cluster 2023-12-07T17:11:46.376271+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:48.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:47 smithi064 bash[25056]: cluster 2023-12-07T17:11:47.632667+0000 mon.a (mon.0) 492 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2023-12-07T17:11:48.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:47 smithi064 bash[36207]: cluster 2023-12-07T17:11:46.376271+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:48.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:47 smithi064 bash[36207]: cluster 2023-12-07T17:11:47.632667+0000 mon.a (mon.0) 492 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2023-12-07T17:11:49.936 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:11:49.964 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:49 smithi184 bash[31336]: cluster 2023-12-07T17:11:48.376883+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:49.964 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:49 smithi184 bash[31336]: cluster 2023-12-07T17:11:48.661712+0000 mon.a (mon.0) 493 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2023-12-07T17:11:50.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:49 smithi064 bash[25056]: cluster 2023-12-07T17:11:48.376883+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:50.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:49 smithi064 bash[25056]: cluster 2023-12-07T17:11:48.661712+0000 mon.a (mon.0) 493 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2023-12-07T17:11:50.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:49 smithi064 bash[36207]: cluster 2023-12-07T17:11:48.376883+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:50.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:49 smithi064 bash[36207]: cluster 2023-12-07T17:11:48.661712+0000 mon.a (mon.0) 493 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2023-12-07T17:11:50.990 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:50 smithi184 bash[31336]: cluster 2023-12-07T17:11:49.663881+0000 mon.a (mon.0) 494 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2023-12-07T17:11:50.990 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:50 smithi184 bash[31336]: cluster 2023-12-07T17:11:50.377426+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:51.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:50 smithi064 bash[25056]: cluster 2023-12-07T17:11:49.663881+0000 mon.a (mon.0) 494 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2023-12-07T17:11:51.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:50 smithi064 bash[25056]: cluster 2023-12-07T17:11:50.377426+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:51.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:50 smithi064 bash[36207]: cluster 2023-12-07T17:11:49.663881+0000 mon.a (mon.0) 494 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2023-12-07T17:11:51.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:50 smithi064 bash[36207]: cluster 2023-12-07T17:11:50.377426+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T17:11:53.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:53 smithi184 bash[31336]: cluster 2023-12-07T17:11:52.378012+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-12-07T17:11:53.867 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:53 smithi064 bash[36207]: cluster 2023-12-07T17:11:52.378012+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-12-07T17:11:53.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:53 smithi064 bash[25056]: cluster 2023-12-07T17:11:52.378012+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/nvme3n1 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-12-07T17:11:54.245 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0173389 s, 605 MB/s 2023-12-07T17:11:54.246 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-12-07T17:11:54.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi184:/dev/nvme3n1 2023-12-07T17:11:55.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:55 smithi184 bash[31336]: cluster 2023-12-07T17:11:54.378633+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-12-07T17:11:55.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:55 smithi064 bash[36207]: cluster 2023-12-07T17:11:54.378633+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-12-07T17:11:55.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:55 smithi064 bash[25056]: cluster 2023-12-07T17:11:54.378633+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: cluster 2023-12-07T17:11:56.379160+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: cephadm 2023-12-07T17:11:56.588192+0000 mgr.y (mgr.14152) 245 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.597005+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.603319+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.604762+0000 mon.a (mon.0) 497 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:57.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: cephadm 2023-12-07T17:11:56.605777+0000 mgr.y (mgr.14152) 246 : cephadm 1 Adjusting osd_memory_target on smithi184 to 17265M 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.610295+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.613243+0000 mon.a (mon.0) 499 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.614914+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[25056]: audit 2023-12-07T17:11:56.626673+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: cluster 2023-12-07T17:11:56.379160+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: cephadm 2023-12-07T17:11:56.588192+0000 mgr.y (mgr.14152) 245 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.597005+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.603319+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.604762+0000 mon.a (mon.0) 497 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:57.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: cephadm 2023-12-07T17:11:56.605777+0000 mgr.y (mgr.14152) 246 : cephadm 1 Adjusting osd_memory_target on smithi184 to 17265M 2023-12-07T17:11:57.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.610295+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:57.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.613243+0000 mon.a (mon.0) 499 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:57.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.614914+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:57.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:57 smithi064 bash[36207]: audit 2023-12-07T17:11:56.626673+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: cluster 2023-12-07T17:11:56.379160+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: cephadm 2023-12-07T17:11:56.588192+0000 mgr.y (mgr.14152) 245 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.597005+0000 mon.a (mon.0) 495 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.603319+0000 mon.a (mon.0) 496 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.604762+0000 mon.a (mon.0) 497 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: cephadm 2023-12-07T17:11:56.605777+0000 mgr.y (mgr.14152) 246 : cephadm 1 Adjusting osd_memory_target on smithi184 to 17265M 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.610295+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.613243+0000 mon.a (mon.0) 499 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.614914+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:11:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:57 smithi184 bash[31336]: audit 2023-12-07T17:11:56.626673+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:11:59.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:11:59 smithi064 bash[25056]: cluster 2023-12-07T17:11:58.379743+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:11:59.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:11:59 smithi064 bash[36207]: cluster 2023-12-07T17:11:58.379743+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:00.060 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:12:00.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:11:59 smithi184 bash[31336]: cluster 2023-12-07T17:11:58.379743+0000 mgr.y (mgr.14152) 247 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:02.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:01 smithi184 bash[31336]: cluster 2023-12-07T17:12:00.380290+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:02.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:01 smithi064 bash[25056]: cluster 2023-12-07T17:12:00.380290+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:02.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:01 smithi064 bash[36207]: cluster 2023-12-07T17:12:00.380290+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:04.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:03 smithi184 bash[31336]: cluster 2023-12-07T17:12:02.380833+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:04.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:03 smithi184 bash[31336]: audit 2023-12-07T17:12:02.862951+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:04.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:03 smithi184 bash[31336]: audit 2023-12-07T17:12:02.869625+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:04.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:03 smithi184 bash[31336]: audit 2023-12-07T17:12:02.871099+0000 mon.a (mon.0) 504 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[25056]: cluster 2023-12-07T17:12:02.380833+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[25056]: audit 2023-12-07T17:12:02.862951+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[25056]: audit 2023-12-07T17:12:02.869625+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[25056]: audit 2023-12-07T17:12:02.871099+0000 mon.a (mon.0) 504 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[36207]: cluster 2023-12-07T17:12:02.380833+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:04.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[36207]: audit 2023-12-07T17:12:02.862951+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:04.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[36207]: audit 2023-12-07T17:12:02.869625+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:04.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:03 smithi064 bash[36207]: audit 2023-12-07T17:12:02.871099+0000 mon.a (mon.0) 504 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:05.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:04 smithi184 bash[31336]: audit 2023-12-07T17:12:02.855632+0000 mgr.y (mgr.14152) 250 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:12:05.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:04 smithi064 bash[25056]: audit 2023-12-07T17:12:02.855632+0000 mgr.y (mgr.14152) 250 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:12:05.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:04 smithi064 bash[36207]: audit 2023-12-07T17:12:02.855632+0000 mgr.y (mgr.14152) 250 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:12:06.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:05 smithi184 bash[31336]: cluster 2023-12-07T17:12:04.381390+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:06.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:05 smithi064 bash[25056]: cluster 2023-12-07T17:12:04.381390+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:06.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:05 smithi064 bash[36207]: cluster 2023-12-07T17:12:04.381390+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:08.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:07 smithi184 bash[31336]: cluster 2023-12-07T17:12:06.382035+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:08.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:07 smithi064 bash[25056]: cluster 2023-12-07T17:12:06.382035+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:08.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:07 smithi064 bash[36207]: cluster 2023-12-07T17:12:06.382035+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:09.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:09 smithi184 bash[31336]: cluster 2023-12-07T17:12:08.382607+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:10.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:09 smithi064 bash[25056]: cluster 2023-12-07T17:12:08.382607+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:10.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:09 smithi064 bash[36207]: cluster 2023-12-07T17:12:08.382607+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:11.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: cluster 2023-12-07T17:12:10.383180+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: audit 2023-12-07T17:12:11.314090+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.184:0/1376854660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: audit 2023-12-07T17:12:11.314953+0000 mon.a (mon.0) 505 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: audit 2023-12-07T17:12:11.318484+0000 mon.a (mon.0) 506 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]': finished 2023-12-07T17:12:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: cluster 2023-12-07T17:12:11.326423+0000 mon.a (mon.0) 507 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2023-12-07T17:12:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:11 smithi184 bash[31336]: audit 2023-12-07T17:12:11.327037+0000 mon.a (mon.0) 508 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:12.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: cluster 2023-12-07T17:12:10.383180+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:12.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: audit 2023-12-07T17:12:11.314090+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.184:0/1376854660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: audit 2023-12-07T17:12:11.314953+0000 mon.a (mon.0) 505 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: audit 2023-12-07T17:12:11.318484+0000 mon.a (mon.0) 506 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]': finished 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: cluster 2023-12-07T17:12:11.326423+0000 mon.a (mon.0) 507 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[36207]: audit 2023-12-07T17:12:11.327037+0000 mon.a (mon.0) 508 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: cluster 2023-12-07T17:12:10.383180+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: audit 2023-12-07T17:12:11.314090+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.184:0/1376854660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: audit 2023-12-07T17:12:11.314953+0000 mon.a (mon.0) 505 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]: dispatch 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: audit 2023-12-07T17:12:11.318484+0000 mon.a (mon.0) 506 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2df3256b-b1c8-4b26-82c6-55d945e2506c"}]': finished 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: cluster 2023-12-07T17:12:11.326423+0000 mon.a (mon.0) 507 : cluster 0 osdmap e35: 6 total, 5 up, 6 in 2023-12-07T17:12:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:11 smithi064 bash[25056]: audit 2023-12-07T17:12:11.327037+0000 mon.a (mon.0) 508 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:14.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:13 smithi184 bash[31336]: cluster 2023-12-07T17:12:12.383767+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:14.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:13 smithi184 bash[31336]: audit 2023-12-07T17:12:13.047809+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.184:0/9569003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:12:14.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:13 smithi064 bash[25056]: cluster 2023-12-07T17:12:12.383767+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:14.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:13 smithi064 bash[25056]: audit 2023-12-07T17:12:13.047809+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.184:0/9569003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:12:14.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:13 smithi064 bash[36207]: cluster 2023-12-07T17:12:12.383767+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:14.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:13 smithi064 bash[36207]: audit 2023-12-07T17:12:13.047809+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.184:0/9569003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:12:15.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:15 smithi184 bash[31336]: cluster 2023-12-07T17:12:14.384282+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:16.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:15 smithi064 bash[25056]: cluster 2023-12-07T17:12:14.384282+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:16.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:15 smithi064 bash[36207]: cluster 2023-12-07T17:12:14.384282+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:17.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:16 smithi184 bash[31336]: cluster 2023-12-07T17:12:16.384791+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:17.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:16 smithi064 bash[25056]: cluster 2023-12-07T17:12:16.384791+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:17.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:16 smithi064 bash[36207]: cluster 2023-12-07T17:12:16.384791+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:19.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:19 smithi184 bash[31336]: cluster 2023-12-07T17:12:18.385501+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:19.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:19 smithi064 bash[25056]: cluster 2023-12-07T17:12:18.385501+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:19.868 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:19 smithi064 bash[36207]: cluster 2023-12-07T17:12:18.385501+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:22.042 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:21 smithi184 bash[31336]: cluster 2023-12-07T17:12:20.386128+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:22.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:21 smithi064 bash[25056]: cluster 2023-12-07T17:12:20.386128+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:22.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:21 smithi064 bash[36207]: cluster 2023-12-07T17:12:20.386128+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:24.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:23 smithi184 bash[31336]: cluster 2023-12-07T17:12:22.386803+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:24.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:23 smithi064 bash[25056]: cluster 2023-12-07T17:12:22.386803+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:24.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:23 smithi064 bash[36207]: cluster 2023-12-07T17:12:22.386803+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:26.027 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:25 smithi184 bash[31336]: cluster 2023-12-07T17:12:24.387514+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:26.028 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:25 smithi184 bash[31336]: audit 2023-12-07T17:12:24.857738+0000 mon.a (mon.0) 509 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T17:12:26.028 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:25 smithi184 bash[31336]: audit 2023-12-07T17:12:24.859454+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:26.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[25056]: cluster 2023-12-07T17:12:24.387514+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:26.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[25056]: audit 2023-12-07T17:12:24.857738+0000 mon.a (mon.0) 509 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T17:12:26.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[25056]: audit 2023-12-07T17:12:24.859454+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:26.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[36207]: cluster 2023-12-07T17:12:24.387514+0000 mgr.y (mgr.14152) 261 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:26.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[36207]: audit 2023-12-07T17:12:24.857738+0000 mon.a (mon.0) 509 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T17:12:26.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:25 smithi064 bash[36207]: audit 2023-12-07T17:12:24.859454+0000 mon.a (mon.0) 510 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:26.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:26 smithi184 bash[31336]: cephadm 2023-12-07T17:12:24.861057+0000 mgr.y (mgr.14152) 262 : cephadm 1 Deploying daemon osd.5 on smithi184 2023-12-07T17:12:27.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:26 smithi064 bash[25056]: cephadm 2023-12-07T17:12:24.861057+0000 mgr.y (mgr.14152) 262 : cephadm 1 Deploying daemon osd.5 on smithi184 2023-12-07T17:12:27.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:26 smithi064 bash[36207]: cephadm 2023-12-07T17:12:24.861057+0000 mgr.y (mgr.14152) 262 : cephadm 1 Deploying daemon osd.5 on smithi184 2023-12-07T17:12:28.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:27 smithi184 bash[31336]: cluster 2023-12-07T17:12:26.388076+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:28.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:27 smithi064 bash[25056]: cluster 2023-12-07T17:12:26.388076+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:28.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:27 smithi064 bash[36207]: cluster 2023-12-07T17:12:26.388076+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:29.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:29 smithi184 bash[31336]: cluster 2023-12-07T17:12:28.388750+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:29.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:29 smithi184 bash[31336]: audit 2023-12-07T17:12:29.372739+0000 mon.a (mon.0) 511 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:29.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:29 smithi184 bash[31336]: audit 2023-12-07T17:12:29.378850+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:29.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:29 smithi184 bash[31336]: audit 2023-12-07T17:12:29.397837+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:30.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[25056]: cluster 2023-12-07T17:12:28.388750+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:30.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[25056]: audit 2023-12-07T17:12:29.372739+0000 mon.a (mon.0) 511 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:30.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[25056]: audit 2023-12-07T17:12:29.378850+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:30.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[25056]: audit 2023-12-07T17:12:29.397837+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[36207]: cluster 2023-12-07T17:12:28.388750+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[36207]: audit 2023-12-07T17:12:29.372739+0000 mon.a (mon.0) 511 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[36207]: audit 2023-12-07T17:12:29.378850+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:29 smithi064 bash[36207]: audit 2023-12-07T17:12:29.397837+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:32.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:31 smithi184 bash[31336]: cluster 2023-12-07T17:12:30.389459+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:32.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:31 smithi064 bash[25056]: cluster 2023-12-07T17:12:30.389459+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:32.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:31 smithi064 bash[36207]: cluster 2023-12-07T17:12:30.389459+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:34.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:33 smithi184 bash[31336]: cluster 2023-12-07T17:12:32.390147+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:34.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:33 smithi064 bash[25056]: cluster 2023-12-07T17:12:32.390147+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:34.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:33 smithi064 bash[36207]: cluster 2023-12-07T17:12:32.390147+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: cluster 2023-12-07T17:12:34.390763+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: audit 2023-12-07T17:12:34.793840+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: audit 2023-12-07T17:12:34.800565+0000 mon.a (mon.0) 515 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: audit 2023-12-07T17:12:34.801867+0000 mon.a (mon.0) 516 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: audit 2023-12-07T17:12:34.803072+0000 mon.a (mon.0) 517 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:35.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:35 smithi184 bash[31336]: audit 2023-12-07T17:12:34.810337+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: cluster 2023-12-07T17:12:34.390763+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:36.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: audit 2023-12-07T17:12:34.793840+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: audit 2023-12-07T17:12:34.800565+0000 mon.a (mon.0) 515 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: audit 2023-12-07T17:12:34.801867+0000 mon.a (mon.0) 516 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: audit 2023-12-07T17:12:34.803072+0000 mon.a (mon.0) 517 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[25056]: audit 2023-12-07T17:12:34.810337+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: cluster 2023-12-07T17:12:34.390763+0000 mgr.y (mgr.14152) 267 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: audit 2023-12-07T17:12:34.793840+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: audit 2023-12-07T17:12:34.800565+0000 mon.a (mon.0) 515 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: audit 2023-12-07T17:12:34.801867+0000 mon.a (mon.0) 516 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: audit 2023-12-07T17:12:34.803072+0000 mon.a (mon.0) 517 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:35 smithi064 bash[36207]: audit 2023-12-07T17:12:34.810337+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:37.041 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:36 smithi184 bash[31336]: audit 2023-12-07T17:12:36.319262+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:37.041 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:36 smithi184 bash[31336]: audit 2023-12-07T17:12:36.319871+0000 mon.a (mon.0) 519 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:37.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:36 smithi064 bash[25056]: audit 2023-12-07T17:12:36.319262+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:37.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:36 smithi064 bash[25056]: audit 2023-12-07T17:12:36.319871+0000 mon.a (mon.0) 519 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:37.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:36 smithi064 bash[36207]: audit 2023-12-07T17:12:36.319262+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:37.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:36 smithi064 bash[36207]: audit 2023-12-07T17:12:36.319871+0000 mon.a (mon.0) 519 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T17:12:38.057 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: cluster 2023-12-07T17:12:36.391282+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:38.058 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: audit 2023-12-07T17:12:36.677585+0000 mon.a (mon.0) 520 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T17:12:38.058 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: audit 2023-12-07T17:12:36.681242+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.058 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: cluster 2023-12-07T17:12:36.683447+0000 mon.a (mon.0) 521 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2023-12-07T17:12:38.058 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: audit 2023-12-07T17:12:36.683934+0000 mon.a (mon.0) 522 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:38.058 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:37 smithi184 bash[31336]: audit 2023-12-07T17:12:36.684547+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: cluster 2023-12-07T17:12:36.391282+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:38.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: audit 2023-12-07T17:12:36.677585+0000 mon.a (mon.0) 520 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: audit 2023-12-07T17:12:36.681242+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: cluster 2023-12-07T17:12:36.683447+0000 mon.a (mon.0) 521 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: audit 2023-12-07T17:12:36.683934+0000 mon.a (mon.0) 522 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[36207]: audit 2023-12-07T17:12:36.684547+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: cluster 2023-12-07T17:12:36.391282+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: audit 2023-12-07T17:12:36.677585+0000 mon.a (mon.0) 520 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: audit 2023-12-07T17:12:36.681242+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: cluster 2023-12-07T17:12:36.683447+0000 mon.a (mon.0) 521 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: audit 2023-12-07T17:12:36.683934+0000 mon.a (mon.0) 522 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:37 smithi064 bash[25056]: audit 2023-12-07T17:12:36.684547+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:12:38.587 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:37.682997+0000 mon.a (mon.0) 524 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: cluster 2023-12-07T17:12:37.686759+0000 mon.a (mon.0) 525 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:37.687458+0000 mon.a (mon.0) 526 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:37.693224+0000 mon.a (mon.0) 527 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: cluster 2023-12-07T17:12:38.391857+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:38.544423+0000 mon.a (mon.0) 528 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:38.551222+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:38 smithi184 bash[31336]: audit 2023-12-07T17:12:38.554090+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:39.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:37.682997+0000 mon.a (mon.0) 524 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:12:39.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: cluster 2023-12-07T17:12:37.686759+0000 mon.a (mon.0) 525 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2023-12-07T17:12:39.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:37.687458+0000 mon.a (mon.0) 526 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:37.693224+0000 mon.a (mon.0) 527 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: cluster 2023-12-07T17:12:38.391857+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:38.544423+0000 mon.a (mon.0) 528 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:38.551222+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[25056]: audit 2023-12-07T17:12:38.554090+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:37.682997+0000 mon.a (mon.0) 524 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: cluster 2023-12-07T17:12:37.686759+0000 mon.a (mon.0) 525 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:37.687458+0000 mon.a (mon.0) 526 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:37.693224+0000 mon.a (mon.0) 527 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: cluster 2023-12-07T17:12:38.391857+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:38.544423+0000 mon.a (mon.0) 528 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:38.551222+0000 mon.a (mon.0) 529 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:38 smithi064 bash[36207]: audit 2023-12-07T17:12:38.554090+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:12:39.239 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.5.service 2023-12-07T17:12:39.241 INFO:tasks.cephadm:Deploying osd.6 on smithi184 with /dev/nvme2n1... 2023-12-07T17:12:39.241 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme2n1 2023-12-07T17:12:39.425 INFO:journalctl@ceph.osd.5.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:12:40.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:39 smithi184 bash[31336]: cluster 2023-12-07T17:12:37.349193+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:12:40.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:39 smithi184 bash[31336]: cluster 2023-12-07T17:12:37.349359+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:12:40.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:39 smithi184 bash[31336]: cluster 2023-12-07T17:12:38.703380+0000 mon.a (mon.0) 531 : cluster 1 osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058] boot 2023-12-07T17:12:40.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:39 smithi184 bash[31336]: cluster 2023-12-07T17:12:38.703458+0000 mon.a (mon.0) 532 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2023-12-07T17:12:40.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:39 smithi184 bash[31336]: audit 2023-12-07T17:12:38.703787+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:40.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[25056]: cluster 2023-12-07T17:12:37.349193+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:12:40.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[25056]: cluster 2023-12-07T17:12:37.349359+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:12:40.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[25056]: cluster 2023-12-07T17:12:38.703380+0000 mon.a (mon.0) 531 : cluster 1 osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058] boot 2023-12-07T17:12:40.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[25056]: cluster 2023-12-07T17:12:38.703458+0000 mon.a (mon.0) 532 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2023-12-07T17:12:40.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[25056]: audit 2023-12-07T17:12:38.703787+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:40.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[36207]: cluster 2023-12-07T17:12:37.349193+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:12:40.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[36207]: cluster 2023-12-07T17:12:37.349359+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:12:40.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[36207]: cluster 2023-12-07T17:12:38.703380+0000 mon.a (mon.0) 531 : cluster 1 osd.5 [v2:172.21.15.184:6808/911723058,v1:172.21.15.184:6809/911723058] boot 2023-12-07T17:12:40.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[36207]: cluster 2023-12-07T17:12:38.703458+0000 mon.a (mon.0) 532 : cluster 0 osdmap e38: 6 total, 6 up, 6 in 2023-12-07T17:12:40.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:39 smithi064 bash[36207]: audit 2023-12-07T17:12:38.703787+0000 mon.a (mon.0) 533 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:12:41.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:40 smithi184 bash[31336]: cluster 2023-12-07T17:12:39.700188+0000 mon.a (mon.0) 534 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2023-12-07T17:12:41.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:40 smithi184 bash[31336]: cluster 2023-12-07T17:12:40.392423+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:41.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:40 smithi064 bash[25056]: cluster 2023-12-07T17:12:39.700188+0000 mon.a (mon.0) 534 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2023-12-07T17:12:41.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:40 smithi064 bash[25056]: cluster 2023-12-07T17:12:40.392423+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:41.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:40 smithi064 bash[36207]: cluster 2023-12-07T17:12:39.700188+0000 mon.a (mon.0) 534 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2023-12-07T17:12:41.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:40 smithi064 bash[36207]: cluster 2023-12-07T17:12:40.392423+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:42.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:41 smithi184 bash[31336]: cluster 2023-12-07T17:12:40.703886+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2023-12-07T17:12:42.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:41 smithi064 bash[25056]: cluster 2023-12-07T17:12:40.703886+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2023-12-07T17:12:42.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:41 smithi064 bash[36207]: cluster 2023-12-07T17:12:40.703886+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2023-12-07T17:12:43.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:42 smithi184 bash[31336]: cluster 2023-12-07T17:12:41.722720+0000 mon.a (mon.0) 536 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2023-12-07T17:12:43.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:42 smithi184 bash[31336]: cluster 2023-12-07T17:12:42.393058+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:43.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:42 smithi064 bash[36207]: cluster 2023-12-07T17:12:41.722720+0000 mon.a (mon.0) 536 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2023-12-07T17:12:43.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:42 smithi064 bash[36207]: cluster 2023-12-07T17:12:42.393058+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:43.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:42 smithi064 bash[25056]: cluster 2023-12-07T17:12:41.722720+0000 mon.a (mon.0) 536 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2023-12-07T17:12:43.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:42 smithi064 bash[25056]: cluster 2023-12-07T17:12:42.393058+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:43.513 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:12:43.815 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:43 smithi184 bash[31336]: cluster 2023-12-07T17:12:42.727786+0000 mon.a (mon.0) 537 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2023-12-07T17:12:44.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:43 smithi064 bash[36207]: cluster 2023-12-07T17:12:42.727786+0000 mon.a (mon.0) 537 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2023-12-07T17:12:44.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:43 smithi064 bash[25056]: cluster 2023-12-07T17:12:42.727786+0000 mon.a (mon.0) 537 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2023-12-07T17:12:44.980 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:44 smithi184 bash[31336]: cluster 2023-12-07T17:12:44.393663+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:45.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:44 smithi064 bash[36207]: cluster 2023-12-07T17:12:44.393663+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:45.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:44 smithi064 bash[25056]: cluster 2023-12-07T17:12:44.393663+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:48.047 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/nvme2n1 2023-12-07T17:12:48.047 INFO:teuthology.orchestra.run.smithi184.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:12:48.047 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2023-12-07T17:12:48.047 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-12-07T17:12:48.047 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-12-07T17:12:48.048 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.018197 s, 576 MB/s 2023-12-07T17:12:48.048 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-12-07T17:12:48.065 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:47 smithi184 bash[31336]: cluster 2023-12-07T17:12:46.394250+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:48.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:47 smithi064 bash[25056]: cluster 2023-12-07T17:12:46.394250+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:48.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:47 smithi064 bash[36207]: cluster 2023-12-07T17:12:46.394250+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:48.792 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi184:/dev/nvme2n1 2023-12-07T17:12:49.957 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:49 smithi184 bash[31336]: cluster 2023-12-07T17:12:48.394794+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:50.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:49 smithi064 bash[25056]: cluster 2023-12-07T17:12:48.394794+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:50.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:49 smithi064 bash[36207]: cluster 2023-12-07T17:12:48.394794+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: cluster 2023-12-07T17:12:50.395367+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: cephadm 2023-12-07T17:12:50.653398+0000 mgr.y (mgr.14152) 276 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.662614+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.668840+0000 mon.a (mon.0) 539 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.670373+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.671621+0000 mon.a (mon.0) 541 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: cephadm 2023-12-07T17:12:50.672629+0000 mgr.y (mgr.14152) 277 : cephadm 1 Adjusting osd_memory_target on smithi184 to 8632M 2023-12-07T17:12:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.677021+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.680037+0000 mon.a (mon.0) 543 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.681479+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:51 smithi184 bash[31336]: audit 2023-12-07T17:12:50.688035+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: cluster 2023-12-07T17:12:50.395367+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: cephadm 2023-12-07T17:12:50.653398+0000 mgr.y (mgr.14152) 276 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.662614+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.668840+0000 mon.a (mon.0) 539 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.670373+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.671621+0000 mon.a (mon.0) 541 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: cephadm 2023-12-07T17:12:50.672629+0000 mgr.y (mgr.14152) 277 : cephadm 1 Adjusting osd_memory_target on smithi184 to 8632M 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.677021+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.680037+0000 mon.a (mon.0) 543 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.681479+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[25056]: audit 2023-12-07T17:12:50.688035+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: cluster 2023-12-07T17:12:50.395367+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: cephadm 2023-12-07T17:12:50.653398+0000 mgr.y (mgr.14152) 276 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.662614+0000 mon.a (mon.0) 538 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.668840+0000 mon.a (mon.0) 539 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.670373+0000 mon.a (mon.0) 540 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.671621+0000 mon.a (mon.0) 541 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: cephadm 2023-12-07T17:12:50.672629+0000 mgr.y (mgr.14152) 277 : cephadm 1 Adjusting osd_memory_target on smithi184 to 8632M 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.677021+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.680037+0000 mon.a (mon.0) 543 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.681479+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:12:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:51 smithi064 bash[36207]: audit 2023-12-07T17:12:50.688035+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:12:54.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:53 smithi184 bash[31336]: cluster 2023-12-07T17:12:52.395936+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:54.101 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:12:54.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:53 smithi064 bash[25056]: cluster 2023-12-07T17:12:52.395936+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:54.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:53 smithi064 bash[36207]: cluster 2023-12-07T17:12:52.395936+0000 mgr.y (mgr.14152) 278 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:56.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:55 smithi184 bash[31336]: cluster 2023-12-07T17:12:54.396492+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:56.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:55 smithi064 bash[25056]: cluster 2023-12-07T17:12:54.396492+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:56.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:55 smithi064 bash[36207]: cluster 2023-12-07T17:12:54.396492+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:57 smithi184 bash[31336]: cluster 2023-12-07T17:12:56.397073+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:57 smithi184 bash[31336]: audit 2023-12-07T17:12:56.900878+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:57 smithi184 bash[31336]: audit 2023-12-07T17:12:56.906337+0000 mon.a (mon.0) 547 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:57 smithi184 bash[31336]: audit 2023-12-07T17:12:56.907632+0000 mon.a (mon.0) 548 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:58.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[25056]: cluster 2023-12-07T17:12:56.397073+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[25056]: audit 2023-12-07T17:12:56.900878+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[25056]: audit 2023-12-07T17:12:56.906337+0000 mon.a (mon.0) 547 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[25056]: audit 2023-12-07T17:12:56.907632+0000 mon.a (mon.0) 548 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[36207]: cluster 2023-12-07T17:12:56.397073+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[36207]: audit 2023-12-07T17:12:56.900878+0000 mon.a (mon.0) 546 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[36207]: audit 2023-12-07T17:12:56.906337+0000 mon.a (mon.0) 547 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:12:58.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:57 smithi064 bash[36207]: audit 2023-12-07T17:12:56.907632+0000 mon.a (mon.0) 548 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:12:59.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:58 smithi184 bash[31336]: audit 2023-12-07T17:12:56.894558+0000 mgr.y (mgr.14152) 281 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:12:59.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:58 smithi064 bash[25056]: audit 2023-12-07T17:12:56.894558+0000 mgr.y (mgr.14152) 281 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:12:59.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:58 smithi064 bash[36207]: audit 2023-12-07T17:12:56.894558+0000 mgr.y (mgr.14152) 281 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:13:00.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:12:59 smithi184 bash[31336]: cluster 2023-12-07T17:12:58.397634+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:00.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:12:59 smithi064 bash[25056]: cluster 2023-12-07T17:12:58.397634+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:00.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:12:59 smithi064 bash[36207]: cluster 2023-12-07T17:12:58.397634+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:01.899 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:01 smithi184 bash[31336]: cluster 2023-12-07T17:13:00.398191+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:02.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:01 smithi064 bash[25056]: cluster 2023-12-07T17:13:00.398191+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:02.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:01 smithi064 bash[36207]: cluster 2023-12-07T17:13:00.398191+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:04.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:03 smithi184 bash[31336]: cluster 2023-12-07T17:13:02.398858+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:04.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:03 smithi064 bash[25056]: cluster 2023-12-07T17:13:02.398858+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:04.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:03 smithi064 bash[36207]: cluster 2023-12-07T17:13:02.398858+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:05.904 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: cluster 2023-12-07T17:13:04.399568+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:05.904 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: audit 2023-12-07T17:13:05.539872+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.184:0/369470327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:05.904 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: audit 2023-12-07T17:13:05.540649+0000 mon.a (mon.0) 549 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:05.905 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: audit 2023-12-07T17:13:05.544372+0000 mon.a (mon.0) 550 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]': finished 2023-12-07T17:13:05.905 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: cluster 2023-12-07T17:13:05.550534+0000 mon.a (mon.0) 551 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2023-12-07T17:13:05.905 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:05 smithi184 bash[31336]: audit 2023-12-07T17:13:05.550960+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:06.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: cluster 2023-12-07T17:13:04.399568+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: audit 2023-12-07T17:13:05.539872+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.184:0/369470327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: audit 2023-12-07T17:13:05.540649+0000 mon.a (mon.0) 549 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: audit 2023-12-07T17:13:05.544372+0000 mon.a (mon.0) 550 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]': finished 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: cluster 2023-12-07T17:13:05.550534+0000 mon.a (mon.0) 551 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[25056]: audit 2023-12-07T17:13:05.550960+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: cluster 2023-12-07T17:13:04.399568+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: audit 2023-12-07T17:13:05.539872+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.184:0/369470327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: audit 2023-12-07T17:13:05.540649+0000 mon.a (mon.0) 549 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]: dispatch 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: audit 2023-12-07T17:13:05.544372+0000 mon.a (mon.0) 550 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ae74eb7-8166-414d-abc5-7f0ecb5f4fce"}]': finished 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: cluster 2023-12-07T17:13:05.550534+0000 mon.a (mon.0) 551 : cluster 0 osdmap e43: 7 total, 6 up, 7 in 2023-12-07T17:13:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:05 smithi064 bash[36207]: audit 2023-12-07T17:13:05.550960+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:08.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:07 smithi184 bash[31336]: cluster 2023-12-07T17:13:06.400161+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:08.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:07 smithi184 bash[31336]: audit 2023-12-07T17:13:07.272886+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.184:0/874681564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:13:08.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:07 smithi064 bash[25056]: cluster 2023-12-07T17:13:06.400161+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:08.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:07 smithi064 bash[25056]: audit 2023-12-07T17:13:07.272886+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.184:0/874681564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:13:08.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:07 smithi064 bash[36207]: cluster 2023-12-07T17:13:06.400161+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:08.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:07 smithi064 bash[36207]: audit 2023-12-07T17:13:07.272886+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.184:0/874681564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:13:10.018 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:09 smithi184 bash[31336]: cluster 2023-12-07T17:13:08.400604+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:10.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:09 smithi064 bash[25056]: cluster 2023-12-07T17:13:08.400604+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:10.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:09 smithi064 bash[36207]: cluster 2023-12-07T17:13:08.400604+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:12.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:11 smithi184 bash[31336]: cluster 2023-12-07T17:13:10.401138+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:12.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:11 smithi064 bash[25056]: cluster 2023-12-07T17:13:10.401138+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:12.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:11 smithi064 bash[36207]: cluster 2023-12-07T17:13:10.401138+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:14.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:13 smithi184 bash[31336]: cluster 2023-12-07T17:13:12.401684+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:14.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:13 smithi064 bash[25056]: cluster 2023-12-07T17:13:12.401684+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:14.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:13 smithi064 bash[36207]: cluster 2023-12-07T17:13:12.401684+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:15.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:15 smithi184 bash[31336]: cluster 2023-12-07T17:13:14.402296+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:16.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:15 smithi064 bash[25056]: cluster 2023-12-07T17:13:14.402296+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:16.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:15 smithi064 bash[36207]: cluster 2023-12-07T17:13:14.402296+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:18.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:17 smithi184 bash[31336]: cluster 2023-12-07T17:13:16.402860+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:18.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:17 smithi064 bash[25056]: cluster 2023-12-07T17:13:16.402860+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:18.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:17 smithi064 bash[36207]: cluster 2023-12-07T17:13:16.402860+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:19.942 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:19 smithi184 bash[31336]: cluster 2023-12-07T17:13:18.403435+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:19.942 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:19 smithi184 bash[31336]: audit 2023-12-07T17:13:19.449138+0000 mon.a (mon.0) 553 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T17:13:19.943 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:19 smithi184 bash[31336]: audit 2023-12-07T17:13:19.450946+0000 mon.a (mon.0) 554 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:20.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[25056]: cluster 2023-12-07T17:13:18.403435+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:20.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[25056]: audit 2023-12-07T17:13:19.449138+0000 mon.a (mon.0) 553 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T17:13:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[25056]: audit 2023-12-07T17:13:19.450946+0000 mon.a (mon.0) 554 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[36207]: cluster 2023-12-07T17:13:18.403435+0000 mgr.y (mgr.14152) 292 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[36207]: audit 2023-12-07T17:13:19.449138+0000 mon.a (mon.0) 553 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T17:13:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:19 smithi064 bash[36207]: audit 2023-12-07T17:13:19.450946+0000 mon.a (mon.0) 554 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:21.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:20 smithi184 bash[31336]: cephadm 2023-12-07T17:13:19.452205+0000 mgr.y (mgr.14152) 293 : cephadm 1 Deploying daemon osd.6 on smithi184 2023-12-07T17:13:21.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:20 smithi064 bash[25056]: cephadm 2023-12-07T17:13:19.452205+0000 mgr.y (mgr.14152) 293 : cephadm 1 Deploying daemon osd.6 on smithi184 2023-12-07T17:13:21.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:20 smithi064 bash[36207]: cephadm 2023-12-07T17:13:19.452205+0000 mgr.y (mgr.14152) 293 : cephadm 1 Deploying daemon osd.6 on smithi184 2023-12-07T17:13:22.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:21 smithi184 bash[31336]: cluster 2023-12-07T17:13:20.404052+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:22.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:21 smithi064 bash[25056]: cluster 2023-12-07T17:13:20.404052+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:22.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:21 smithi064 bash[36207]: cluster 2023-12-07T17:13:20.404052+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:23.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:22 smithi184 bash[31336]: cluster 2023-12-07T17:13:22.404568+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:23.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:22 smithi064 bash[25056]: cluster 2023-12-07T17:13:22.404568+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:23.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:22 smithi064 bash[36207]: cluster 2023-12-07T17:13:22.404568+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:25.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:25 smithi184 bash[31336]: audit 2023-12-07T17:13:24.186000+0000 mon.a (mon.0) 555 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:25 smithi184 bash[31336]: audit 2023-12-07T17:13:24.191942+0000 mon.a (mon.0) 556 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:25 smithi184 bash[31336]: audit 2023-12-07T17:13:24.212586+0000 mon.a (mon.0) 557 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:25.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:25 smithi184 bash[31336]: cluster 2023-12-07T17:13:24.405137+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[25056]: audit 2023-12-07T17:13:24.186000+0000 mon.a (mon.0) 555 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[25056]: audit 2023-12-07T17:13:24.191942+0000 mon.a (mon.0) 556 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[25056]: audit 2023-12-07T17:13:24.212586+0000 mon.a (mon.0) 557 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[25056]: cluster 2023-12-07T17:13:24.405137+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[36207]: audit 2023-12-07T17:13:24.186000+0000 mon.a (mon.0) 555 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[36207]: audit 2023-12-07T17:13:24.191942+0000 mon.a (mon.0) 556 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[36207]: audit 2023-12-07T17:13:24.212586+0000 mon.a (mon.0) 557 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:25.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:25 smithi064 bash[36207]: cluster 2023-12-07T17:13:24.405137+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:27.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:27 smithi184 bash[31336]: cluster 2023-12-07T17:13:26.405710+0000 mgr.y (mgr.14152) 297 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:28.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:27 smithi064 bash[25056]: cluster 2023-12-07T17:13:26.405710+0000 mgr.y (mgr.14152) 297 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:28.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:27 smithi064 bash[36207]: cluster 2023-12-07T17:13:26.405710+0000 mgr.y (mgr.14152) 297 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:30.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:29 smithi184 bash[31336]: cluster 2023-12-07T17:13:28.406264+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:30.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:29 smithi184 bash[31336]: audit 2023-12-07T17:13:29.609068+0000 mon.a (mon.0) 558 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:29 smithi184 bash[31336]: audit 2023-12-07T17:13:29.617178+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[25056]: cluster 2023-12-07T17:13:28.406264+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[25056]: audit 2023-12-07T17:13:29.609068+0000 mon.a (mon.0) 558 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[25056]: audit 2023-12-07T17:13:29.617178+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[36207]: cluster 2023-12-07T17:13:28.406264+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[36207]: audit 2023-12-07T17:13:29.609068+0000 mon.a (mon.0) 558 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:29 smithi064 bash[36207]: audit 2023-12-07T17:13:29.617178+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:30.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:30 smithi184 bash[31336]: audit 2023-12-07T17:13:30.244862+0000 mon.a (mon.0) 560 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:30.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:30 smithi184 bash[31336]: audit 2023-12-07T17:13:30.247085+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:30.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:30 smithi184 bash[31336]: audit 2023-12-07T17:13:30.256781+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:31.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[25056]: audit 2023-12-07T17:13:30.244862+0000 mon.a (mon.0) 560 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:31.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[25056]: audit 2023-12-07T17:13:30.247085+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:31.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[25056]: audit 2023-12-07T17:13:30.256781+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:31.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[36207]: audit 2023-12-07T17:13:30.244862+0000 mon.a (mon.0) 560 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:31.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[36207]: audit 2023-12-07T17:13:30.247085+0000 mon.a (mon.0) 561 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:31.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:30 smithi064 bash[36207]: audit 2023-12-07T17:13:30.256781+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:32.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:31 smithi184 bash[31336]: cluster 2023-12-07T17:13:30.406855+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:32.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:31 smithi184 bash[31336]: audit 2023-12-07T17:13:31.157096+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:32.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:31 smithi184 bash[31336]: audit 2023-12-07T17:13:31.157864+0000 mon.a (mon.0) 563 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:32.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[25056]: cluster 2023-12-07T17:13:30.406855+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:32.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[25056]: audit 2023-12-07T17:13:31.157096+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:32.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[25056]: audit 2023-12-07T17:13:31.157864+0000 mon.a (mon.0) 563 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:32.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[36207]: cluster 2023-12-07T17:13:30.406855+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:32.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[36207]: audit 2023-12-07T17:13:31.157096+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:32.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:31 smithi064 bash[36207]: audit 2023-12-07T17:13:31.157864+0000 mon.a (mon.0) 563 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T17:13:33.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:32 smithi184 bash[31336]: audit 2023-12-07T17:13:31.665864+0000 mon.a (mon.0) 564 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T17:13:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:32 smithi184 bash[31336]: cluster 2023-12-07T17:13:31.668858+0000 mon.a (mon.0) 565 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2023-12-07T17:13:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:32 smithi184 bash[31336]: audit 2023-12-07T17:13:31.669126+0000 mon.a (mon.0) 566 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:32 smithi184 bash[31336]: audit 2023-12-07T17:13:31.669445+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:32 smithi184 bash[31336]: audit 2023-12-07T17:13:31.669718+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[25056]: audit 2023-12-07T17:13:31.665864+0000 mon.a (mon.0) 564 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T17:13:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[25056]: cluster 2023-12-07T17:13:31.668858+0000 mon.a (mon.0) 565 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2023-12-07T17:13:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[25056]: audit 2023-12-07T17:13:31.669126+0000 mon.a (mon.0) 566 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[25056]: audit 2023-12-07T17:13:31.669445+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[25056]: audit 2023-12-07T17:13:31.669718+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[36207]: audit 2023-12-07T17:13:31.665864+0000 mon.a (mon.0) 564 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T17:13:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[36207]: cluster 2023-12-07T17:13:31.668858+0000 mon.a (mon.0) 565 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2023-12-07T17:13:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[36207]: audit 2023-12-07T17:13:31.669126+0000 mon.a (mon.0) 566 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[36207]: audit 2023-12-07T17:13:31.669445+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:32 smithi064 bash[36207]: audit 2023-12-07T17:13:31.669718+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:13:33.324 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 6 on host 'smithi184' 2023-12-07T17:13:34.042 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: cluster 2023-12-07T17:13:32.407391+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:34.042 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:32.668556+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:13:34.042 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: cluster 2023-12-07T17:13:32.675302+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2023-12-07T17:13:34.042 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:32.676185+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:32.678539+0000 mon.a (mon.0) 571 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:33.299453+0000 mon.a (mon.0) 572 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:33.305643+0000 mon.a (mon.0) 573 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:33.314573+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: cluster 2023-12-07T17:13:33.676715+0000 mon.a (mon.0) 575 : cluster 1 osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913] boot 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: cluster 2023-12-07T17:13:33.676795+0000 mon.a (mon.0) 576 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2023-12-07T17:13:34.043 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:33 smithi184 bash[31336]: audit 2023-12-07T17:13:33.677406+0000 mon.a (mon.0) 577 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.044 DEBUG:teuthology.orchestra.run.smithi184:osd.6> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.6.service 2023-12-07T17:13:34.046 INFO:tasks.cephadm:Deploying osd.7 on smithi184 with /dev/nvme1n1... 2023-12-07T17:13:34.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/nvme1n1 2023-12-07T17:13:34.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: cluster 2023-12-07T17:13:32.407391+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:34.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:32.668556+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:13:34.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: cluster 2023-12-07T17:13:32.675302+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2023-12-07T17:13:34.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:32.676185+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:32.678539+0000 mon.a (mon.0) 571 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:33.299453+0000 mon.a (mon.0) 572 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:33.305643+0000 mon.a (mon.0) 573 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:33.314573+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: cluster 2023-12-07T17:13:33.676715+0000 mon.a (mon.0) 575 : cluster 1 osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913] boot 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: cluster 2023-12-07T17:13:33.676795+0000 mon.a (mon.0) 576 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2023-12-07T17:13:34.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[25056]: audit 2023-12-07T17:13:33.677406+0000 mon.a (mon.0) 577 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: cluster 2023-12-07T17:13:32.407391+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:32.668556+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: cluster 2023-12-07T17:13:32.675302+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:32.676185+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:32.678539+0000 mon.a (mon.0) 571 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:33.299453+0000 mon.a (mon.0) 572 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:33.305643+0000 mon.a (mon.0) 573 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:33.314573+0000 mon.a (mon.0) 574 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: cluster 2023-12-07T17:13:33.676715+0000 mon.a (mon.0) 575 : cluster 1 osd.6 [v2:172.21.15.184:6816/3929847913,v1:172.21.15.184:6817/3929847913] boot 2023-12-07T17:13:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: cluster 2023-12-07T17:13:33.676795+0000 mon.a (mon.0) 576 : cluster 0 osdmap e46: 7 total, 7 up, 7 in 2023-12-07T17:13:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:33 smithi064 bash[36207]: audit 2023-12-07T17:13:33.677406+0000 mon.a (mon.0) 577 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:13:34.406 INFO:journalctl@ceph.osd.6.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:13:35.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:34 smithi184 bash[31336]: cluster 2023-12-07T17:13:32.110351+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:13:35.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:34 smithi184 bash[31336]: cluster 2023-12-07T17:13:32.110452+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:13:35.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:34 smithi184 bash[31336]: cluster 2023-12-07T17:13:34.678670+0000 mon.a (mon.0) 578 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2023-12-07T17:13:35.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[25056]: cluster 2023-12-07T17:13:32.110351+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:13:35.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[25056]: cluster 2023-12-07T17:13:32.110452+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:13:35.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[25056]: cluster 2023-12-07T17:13:34.678670+0000 mon.a (mon.0) 578 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2023-12-07T17:13:35.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[36207]: cluster 2023-12-07T17:13:32.110351+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:13:35.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[36207]: cluster 2023-12-07T17:13:32.110452+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:13:35.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:34 smithi064 bash[36207]: cluster 2023-12-07T17:13:34.678670+0000 mon.a (mon.0) 578 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2023-12-07T17:13:36.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:35 smithi184 bash[31336]: cluster 2023-12-07T17:13:34.407973+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:36.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:35 smithi184 bash[31336]: cluster 2023-12-07T17:13:35.682995+0000 mon.a (mon.0) 579 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2023-12-07T17:13:36.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:35 smithi064 bash[25056]: cluster 2023-12-07T17:13:34.407973+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:36.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:35 smithi064 bash[25056]: cluster 2023-12-07T17:13:35.682995+0000 mon.a (mon.0) 579 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2023-12-07T17:13:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:35 smithi064 bash[36207]: cluster 2023-12-07T17:13:34.407973+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:36.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:35 smithi064 bash[36207]: cluster 2023-12-07T17:13:35.682995+0000 mon.a (mon.0) 579 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2023-12-07T17:13:37.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:36 smithi184 bash[31336]: cluster 2023-12-07T17:13:36.408513+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:37.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:36 smithi064 bash[25056]: cluster 2023-12-07T17:13:36.408513+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:37.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:36 smithi064 bash[36207]: cluster 2023-12-07T17:13:36.408513+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:38.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:37 smithi184 bash[31336]: cluster 2023-12-07T17:13:36.704466+0000 mon.a (mon.0) 580 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2023-12-07T17:13:38.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:37 smithi064 bash[25056]: cluster 2023-12-07T17:13:36.704466+0000 mon.a (mon.0) 580 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2023-12-07T17:13:38.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:37 smithi064 bash[36207]: cluster 2023-12-07T17:13:36.704466+0000 mon.a (mon.0) 580 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2023-12-07T17:13:38.266 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:13:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:38 smithi184 bash[31336]: cluster 2023-12-07T17:13:37.713419+0000 mon.a (mon.0) 581 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2023-12-07T17:13:39.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:38 smithi184 bash[31336]: cluster 2023-12-07T17:13:38.409064+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:39.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:38 smithi064 bash[25056]: cluster 2023-12-07T17:13:37.713419+0000 mon.a (mon.0) 581 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2023-12-07T17:13:39.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:38 smithi064 bash[25056]: cluster 2023-12-07T17:13:38.409064+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:38 smithi064 bash[36207]: cluster 2023-12-07T17:13:37.713419+0000 mon.a (mon.0) 581 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2023-12-07T17:13:39.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:38 smithi064 bash[36207]: cluster 2023-12-07T17:13:38.409064+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:42.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:41 smithi184 bash[31336]: cluster 2023-12-07T17:13:40.409840+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T17:13:42.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:41 smithi064 bash[25056]: cluster 2023-12-07T17:13:40.409840+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T17:13:42.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:41 smithi064 bash[36207]: cluster 2023-12-07T17:13:40.409840+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T17:13:42.554 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/nvme1n1 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0169488 s, 619 MB/s 2023-12-07T17:13:42.555 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-12-07T17:13:43.387 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi184:/dev/nvme1n1 2023-12-07T17:13:44.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:43 smithi184 bash[31336]: cluster 2023-12-07T17:13:42.410504+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T17:13:44.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:43 smithi064 bash[25056]: cluster 2023-12-07T17:13:42.410504+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T17:13:44.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:43 smithi064 bash[36207]: cluster 2023-12-07T17:13:42.410504+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T17:13:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: cluster 2023-12-07T17:13:44.411211+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T17:13:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.066128+0000 mon.a (mon.0) 582 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.075440+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.077320+0000 mon.a (mon.0) 584 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.078837+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.080200+0000 mon.a (mon.0) 586 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.088907+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.092362+0000 mon.a (mon.0) 588 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.093567+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:46.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:45 smithi184 bash[31336]: audit 2023-12-07T17:13:45.103554+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: cluster 2023-12-07T17:13:44.411211+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.066128+0000 mon.a (mon.0) 582 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.075440+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.077320+0000 mon.a (mon.0) 584 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.078837+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.080200+0000 mon.a (mon.0) 586 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.088907+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.092362+0000 mon.a (mon.0) 588 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.093567+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[25056]: audit 2023-12-07T17:13:45.103554+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: cluster 2023-12-07T17:13:44.411211+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.066128+0000 mon.a (mon.0) 582 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.075440+0000 mon.a (mon.0) 583 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.077320+0000 mon.a (mon.0) 584 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.078837+0000 mon.a (mon.0) 585 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.080200+0000 mon.a (mon.0) 586 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.088907+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.092362+0000 mon.a (mon.0) 588 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.093567+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:13:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:45 smithi064 bash[36207]: audit 2023-12-07T17:13:45.103554+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:13:47.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:46 smithi184 bash[31336]: cephadm 2023-12-07T17:13:45.054381+0000 mgr.y (mgr.14152) 307 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:13:47.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:46 smithi184 bash[31336]: cephadm 2023-12-07T17:13:45.081325+0000 mgr.y (mgr.14152) 308 : cephadm 1 Adjusting osd_memory_target on smithi184 to 5755M 2023-12-07T17:13:47.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:46 smithi064 bash[25056]: cephadm 2023-12-07T17:13:45.054381+0000 mgr.y (mgr.14152) 307 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:13:47.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:46 smithi064 bash[25056]: cephadm 2023-12-07T17:13:45.081325+0000 mgr.y (mgr.14152) 308 : cephadm 1 Adjusting osd_memory_target on smithi184 to 5755M 2023-12-07T17:13:47.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:46 smithi064 bash[36207]: cephadm 2023-12-07T17:13:45.054381+0000 mgr.y (mgr.14152) 307 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:13:47.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:46 smithi064 bash[36207]: cephadm 2023-12-07T17:13:45.081325+0000 mgr.y (mgr.14152) 308 : cephadm 1 Adjusting osd_memory_target on smithi184 to 5755M 2023-12-07T17:13:48.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:47 smithi184 bash[31336]: cluster 2023-12-07T17:13:46.411809+0000 mgr.y (mgr.14152) 309 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-07T17:13:48.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:47 smithi064 bash[25056]: cluster 2023-12-07T17:13:46.411809+0000 mgr.y (mgr.14152) 309 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-07T17:13:48.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:47 smithi064 bash[36207]: cluster 2023-12-07T17:13:46.411809+0000 mgr.y (mgr.14152) 309 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-07T17:13:48.551 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:13:50.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:49 smithi184 bash[31336]: cluster 2023-12-07T17:13:48.412387+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-07T17:13:50.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:49 smithi064 bash[25056]: cluster 2023-12-07T17:13:48.412387+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-07T17:13:50.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:49 smithi064 bash[36207]: cluster 2023-12-07T17:13:48.412387+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-07T17:13:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:51 smithi184 bash[31336]: cluster 2023-12-07T17:13:50.412998+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T17:13:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:51 smithi184 bash[31336]: audit 2023-12-07T17:13:51.469018+0000 mon.a (mon.0) 591 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:13:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:51 smithi184 bash[31336]: audit 2023-12-07T17:13:51.475119+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:13:52.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:51 smithi184 bash[31336]: audit 2023-12-07T17:13:51.476455+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[25056]: cluster 2023-12-07T17:13:50.412998+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[25056]: audit 2023-12-07T17:13:51.469018+0000 mon.a (mon.0) 591 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[25056]: audit 2023-12-07T17:13:51.475119+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[25056]: audit 2023-12-07T17:13:51.476455+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[36207]: cluster 2023-12-07T17:13:50.412998+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[36207]: audit 2023-12-07T17:13:51.469018+0000 mon.a (mon.0) 591 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[36207]: audit 2023-12-07T17:13:51.475119+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T17:13:52.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:51 smithi064 bash[36207]: audit 2023-12-07T17:13:51.476455+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:13:53.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:52 smithi184 bash[31336]: audit 2023-12-07T17:13:51.462333+0000 mgr.y (mgr.14152) 312 : audit 0 from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:13:53.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:52 smithi064 bash[25056]: audit 2023-12-07T17:13:51.462333+0000 mgr.y (mgr.14152) 312 : audit 0 from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:13:53.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:52 smithi064 bash[36207]: audit 2023-12-07T17:13:51.462333+0000 mgr.y (mgr.14152) 312 : audit 0 from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:13:54.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:53 smithi184 bash[31336]: cluster 2023-12-07T17:13:52.413661+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:54.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:53 smithi064 bash[25056]: cluster 2023-12-07T17:13:52.413661+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:54.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:53 smithi064 bash[36207]: cluster 2023-12-07T17:13:52.413661+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:55.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:54 smithi184 bash[31336]: cluster 2023-12-07T17:13:54.414403+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:55.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:54 smithi064 bash[25056]: cluster 2023-12-07T17:13:54.414403+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:55.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:54 smithi064 bash[36207]: cluster 2023-12-07T17:13:54.414403+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:58.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:57 smithi184 bash[31336]: cluster 2023-12-07T17:13:56.414930+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:58.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:57 smithi064 bash[25056]: cluster 2023-12-07T17:13:56.414930+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:13:58.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:57 smithi064 bash[36207]: cluster 2023-12-07T17:13:56.414930+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:00.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:13:59 smithi184 bash[31336]: cluster 2023-12-07T17:13:58.415479+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:00.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:13:59 smithi064 bash[25056]: cluster 2023-12-07T17:13:58.415479+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:00.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:13:59 smithi064 bash[36207]: cluster 2023-12-07T17:13:58.415479+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:01.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:00 smithi184 bash[31336]: audit 2023-12-07T17:14:00.146285+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.184:0/254305743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:00 smithi184 bash[31336]: audit 2023-12-07T17:14:00.147156+0000 mon.a (mon.0) 594 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:00 smithi184 bash[31336]: audit 2023-12-07T17:14:00.150962+0000 mon.a (mon.0) 595 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]': finished 2023-12-07T17:14:01.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:00 smithi184 bash[31336]: cluster 2023-12-07T17:14:00.156538+0000 mon.a (mon.0) 596 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2023-12-07T17:14:01.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:00 smithi184 bash[31336]: audit 2023-12-07T17:14:00.157166+0000 mon.a (mon.0) 597 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[36207]: audit 2023-12-07T17:14:00.146285+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.184:0/254305743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[36207]: audit 2023-12-07T17:14:00.147156+0000 mon.a (mon.0) 594 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[36207]: audit 2023-12-07T17:14:00.150962+0000 mon.a (mon.0) 595 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]': finished 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[36207]: cluster 2023-12-07T17:14:00.156538+0000 mon.a (mon.0) 596 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[36207]: audit 2023-12-07T17:14:00.157166+0000 mon.a (mon.0) 597 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[25056]: audit 2023-12-07T17:14:00.146285+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.184:0/254305743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[25056]: audit 2023-12-07T17:14:00.147156+0000 mon.a (mon.0) 594 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]: dispatch 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[25056]: audit 2023-12-07T17:14:00.150962+0000 mon.a (mon.0) 595 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1498aa6-78dd-4a46-9a67-4454cb6f66fb"}]': finished 2023-12-07T17:14:01.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[25056]: cluster 2023-12-07T17:14:00.156538+0000 mon.a (mon.0) 596 : cluster 0 osdmap e51: 8 total, 7 up, 8 in 2023-12-07T17:14:01.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:00 smithi064 bash[25056]: audit 2023-12-07T17:14:00.157166+0000 mon.a (mon.0) 597 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:02.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:01 smithi184 bash[31336]: cluster 2023-12-07T17:14:00.416094+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:02.118 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:01 smithi064 bash[36207]: cluster 2023-12-07T17:14:00.416094+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:02.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:01 smithi064 bash[25056]: cluster 2023-12-07T17:14:00.416094+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:03.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:02 smithi184 bash[31336]: audit 2023-12-07T17:14:01.840156+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.184:0/4071313885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:14:03.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:02 smithi064 bash[36207]: audit 2023-12-07T17:14:01.840156+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.184:0/4071313885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:14:03.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:02 smithi064 bash[25056]: audit 2023-12-07T17:14:01.840156+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.184:0/4071313885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T17:14:04.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:03 smithi184 bash[31336]: cluster 2023-12-07T17:14:02.416656+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:04.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:03 smithi064 bash[36207]: cluster 2023-12-07T17:14:02.416656+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:04.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:03 smithi064 bash[25056]: cluster 2023-12-07T17:14:02.416656+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:06.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:05 smithi184 bash[31336]: cluster 2023-12-07T17:14:04.417378+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:06.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:05 smithi064 bash[25056]: cluster 2023-12-07T17:14:04.417378+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:06.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:05 smithi064 bash[36207]: cluster 2023-12-07T17:14:04.417378+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:08.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:07 smithi184 bash[31336]: cluster 2023-12-07T17:14:06.417994+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:08.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:07 smithi064 bash[25056]: cluster 2023-12-07T17:14:06.417994+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:08.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:07 smithi064 bash[36207]: cluster 2023-12-07T17:14:06.417994+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:10.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:09 smithi184 bash[31336]: cluster 2023-12-07T17:14:08.418568+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:10.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:09 smithi064 bash[25056]: cluster 2023-12-07T17:14:08.418568+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:10.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:09 smithi064 bash[36207]: cluster 2023-12-07T17:14:08.418568+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:11.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:11 smithi184 bash[31336]: cluster 2023-12-07T17:14:10.419161+0000 mgr.y (mgr.14152) 322 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:12.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:11 smithi064 bash[25056]: cluster 2023-12-07T17:14:10.419161+0000 mgr.y (mgr.14152) 322 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:12.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:11 smithi064 bash[36207]: cluster 2023-12-07T17:14:10.419161+0000 mgr.y (mgr.14152) 322 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:14.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:13 smithi184 bash[31336]: cluster 2023-12-07T17:14:12.419765+0000 mgr.y (mgr.14152) 323 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:14.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:13 smithi064 bash[25056]: cluster 2023-12-07T17:14:12.419765+0000 mgr.y (mgr.14152) 323 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:14.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:13 smithi064 bash[36207]: cluster 2023-12-07T17:14:12.419765+0000 mgr.y (mgr.14152) 323 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:14.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:14 smithi184 bash[31336]: audit 2023-12-07T17:14:14.085980+0000 mon.a (mon.0) 598 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T17:14:14.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:14 smithi184 bash[31336]: audit 2023-12-07T17:14:14.087481+0000 mon.a (mon.0) 599 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:15.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:14 smithi064 bash[25056]: audit 2023-12-07T17:14:14.085980+0000 mon.a (mon.0) 598 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T17:14:15.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:14 smithi064 bash[25056]: audit 2023-12-07T17:14:14.087481+0000 mon.a (mon.0) 599 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:15.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:14 smithi064 bash[36207]: audit 2023-12-07T17:14:14.085980+0000 mon.a (mon.0) 598 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T17:14:15.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:14 smithi064 bash[36207]: audit 2023-12-07T17:14:14.087481+0000 mon.a (mon.0) 599 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:16.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:15 smithi184 bash[31336]: cephadm 2023-12-07T17:14:14.088760+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon osd.7 on smithi184 2023-12-07T17:14:16.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:15 smithi184 bash[31336]: cluster 2023-12-07T17:14:14.420340+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:16.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:15 smithi064 bash[25056]: cephadm 2023-12-07T17:14:14.088760+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon osd.7 on smithi184 2023-12-07T17:14:16.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:15 smithi064 bash[25056]: cluster 2023-12-07T17:14:14.420340+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:16.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:15 smithi064 bash[36207]: cephadm 2023-12-07T17:14:14.088760+0000 mgr.y (mgr.14152) 324 : cephadm 1 Deploying daemon osd.7 on smithi184 2023-12-07T17:14:16.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:15 smithi064 bash[36207]: cluster 2023-12-07T17:14:14.420340+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:18.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:17 smithi184 bash[31336]: cluster 2023-12-07T17:14:16.420933+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:18.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:17 smithi064 bash[25056]: cluster 2023-12-07T17:14:16.420933+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:18.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:17 smithi064 bash[36207]: cluster 2023-12-07T17:14:16.420933+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:20.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:19 smithi184 bash[31336]: cluster 2023-12-07T17:14:18.421530+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:20.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:19 smithi184 bash[31336]: audit 2023-12-07T17:14:18.817303+0000 mon.a (mon.0) 600 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:19 smithi184 bash[31336]: audit 2023-12-07T17:14:18.824519+0000 mon.a (mon.0) 601 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:19 smithi184 bash[31336]: audit 2023-12-07T17:14:18.833104+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[25056]: cluster 2023-12-07T17:14:18.421530+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[25056]: audit 2023-12-07T17:14:18.817303+0000 mon.a (mon.0) 600 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[25056]: audit 2023-12-07T17:14:18.824519+0000 mon.a (mon.0) 601 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[25056]: audit 2023-12-07T17:14:18.833104+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[36207]: cluster 2023-12-07T17:14:18.421530+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[36207]: audit 2023-12-07T17:14:18.817303+0000 mon.a (mon.0) 600 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[36207]: audit 2023-12-07T17:14:18.824519+0000 mon.a (mon.0) 601 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:20.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:19 smithi064 bash[36207]: audit 2023-12-07T17:14:18.833104+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:22.009 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:21 smithi184 bash[31336]: cluster 2023-12-07T17:14:20.422133+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:22.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:21 smithi064 bash[25056]: cluster 2023-12-07T17:14:20.422133+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:22.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:21 smithi064 bash[36207]: cluster 2023-12-07T17:14:20.422133+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:23.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:22 smithi184 bash[31336]: cluster 2023-12-07T17:14:22.422691+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:23.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:22 smithi064 bash[25056]: cluster 2023-12-07T17:14:22.422691+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:23.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:22 smithi064 bash[36207]: cluster 2023-12-07T17:14:22.422691+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:25.566 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: audit 2023-12-07T17:14:24.275432+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: audit 2023-12-07T17:14:24.282588+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: audit 2023-12-07T17:14:24.284141+0000 mon.a (mon.0) 605 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:25.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: audit 2023-12-07T17:14:24.285682+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:25.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: audit 2023-12-07T17:14:24.293115+0000 mon.a (mon.0) 607 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:25 smithi184 bash[31336]: cluster 2023-12-07T17:14:24.423279+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:25.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: audit 2023-12-07T17:14:24.275432+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: audit 2023-12-07T17:14:24.282588+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: audit 2023-12-07T17:14:24.284141+0000 mon.a (mon.0) 605 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: audit 2023-12-07T17:14:24.285682+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: audit 2023-12-07T17:14:24.293115+0000 mon.a (mon.0) 607 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[25056]: cluster 2023-12-07T17:14:24.423279+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: audit 2023-12-07T17:14:24.275432+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: audit 2023-12-07T17:14:24.282588+0000 mon.a (mon.0) 604 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: audit 2023-12-07T17:14:24.284141+0000 mon.a (mon.0) 605 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: audit 2023-12-07T17:14:24.285682+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: audit 2023-12-07T17:14:24.293115+0000 mon.a (mon.0) 607 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:25.623 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:25 smithi064 bash[36207]: cluster 2023-12-07T17:14:24.423279+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:26.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:26 smithi184 bash[31336]: audit 2023-12-07T17:14:25.677442+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:26.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:26 smithi184 bash[31336]: audit 2023-12-07T17:14:25.677872+0000 mon.a (mon.0) 608 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:27.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:26 smithi064 bash[25056]: audit 2023-12-07T17:14:25.677442+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:27.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:26 smithi064 bash[25056]: audit 2023-12-07T17:14:25.677872+0000 mon.a (mon.0) 608 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:27.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:26 smithi064 bash[36207]: audit 2023-12-07T17:14:25.677442+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:27.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:26 smithi064 bash[36207]: audit 2023-12-07T17:14:25.677872+0000 mon.a (mon.0) 608 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T17:14:27.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: cluster 2023-12-07T17:14:26.423662+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:27.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: audit 2023-12-07T17:14:26.680071+0000 mon.a (mon.0) 609 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T17:14:27.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: cluster 2023-12-07T17:14:26.683871+0000 mon.a (mon.0) 610 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2023-12-07T17:14:27.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: audit 2023-12-07T17:14:26.684266+0000 mon.a (mon.0) 611 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:27.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: audit 2023-12-07T17:14:26.686248+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:27.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:27 smithi184 bash[31336]: audit 2023-12-07T17:14:26.686661+0000 mon.a (mon.0) 612 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:28.003 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 7 on host 'smithi184' 2023-12-07T17:14:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: cluster 2023-12-07T17:14:26.423662+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: audit 2023-12-07T17:14:26.680071+0000 mon.a (mon.0) 609 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T17:14:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: cluster 2023-12-07T17:14:26.683871+0000 mon.a (mon.0) 610 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2023-12-07T17:14:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: audit 2023-12-07T17:14:26.684266+0000 mon.a (mon.0) 611 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: audit 2023-12-07T17:14:26.686248+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[25056]: audit 2023-12-07T17:14:26.686661+0000 mon.a (mon.0) 612 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: cluster 2023-12-07T17:14:26.423662+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: audit 2023-12-07T17:14:26.680071+0000 mon.a (mon.0) 609 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: cluster 2023-12-07T17:14:26.683871+0000 mon.a (mon.0) 610 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: audit 2023-12-07T17:14:26.684266+0000 mon.a (mon.0) 611 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: audit 2023-12-07T17:14:26.686248+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:27 smithi064 bash[36207]: audit 2023-12-07T17:14:26.686661+0000 mon.a (mon.0) 612 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-12-07T17:14:28.693 DEBUG:teuthology.orchestra.run.smithi184:osd.7> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.7.service 2023-12-07T17:14:28.696 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2023-12-07T17:14:28.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-12-07T17:14:28.865 INFO:journalctl@ceph.osd.7.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: cluster 2023-12-07T17:14:26.660608+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: cluster 2023-12-07T17:14:26.660713+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.682761+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: cluster 2023-12-07T17:14:27.687804+0000 mon.a (mon.0) 614 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.688795+0000 mon.a (mon.0) 615 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.694891+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.966427+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.976682+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:28 smithi184 bash[31336]: audit 2023-12-07T17:14:27.987765+0000 mon.a (mon.0) 619 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: cluster 2023-12-07T17:14:26.660608+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:14:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: cluster 2023-12-07T17:14:26.660713+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.682761+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: cluster 2023-12-07T17:14:27.687804+0000 mon.a (mon.0) 614 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.688795+0000 mon.a (mon.0) 615 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.694891+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.966427+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.976682+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[25056]: audit 2023-12-07T17:14:27.987765+0000 mon.a (mon.0) 619 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: cluster 2023-12-07T17:14:26.660608+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: cluster 2023-12-07T17:14:26.660713+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.682761+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-12-07T17:14:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: cluster 2023-12-07T17:14:27.687804+0000 mon.a (mon.0) 614 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2023-12-07T17:14:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.688795+0000 mon.a (mon.0) 615 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.694891+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.966427+0000 mon.a (mon.0) 617 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.976682+0000 mon.a (mon.0) 618 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:28 smithi064 bash[36207]: audit 2023-12-07T17:14:27.987765+0000 mon.a (mon.0) 619 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:14:30.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:29 smithi184 bash[31336]: cluster 2023-12-07T17:14:28.424200+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:30.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:29 smithi184 bash[31336]: cluster 2023-12-07T17:14:28.691826+0000 mon.a (mon.0) 620 : cluster 1 osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552] boot 2023-12-07T17:14:30.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:29 smithi184 bash[31336]: cluster 2023-12-07T17:14:28.691894+0000 mon.a (mon.0) 621 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2023-12-07T17:14:30.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:29 smithi184 bash[31336]: audit 2023-12-07T17:14:28.692199+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[25056]: cluster 2023-12-07T17:14:28.424200+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[25056]: cluster 2023-12-07T17:14:28.691826+0000 mon.a (mon.0) 620 : cluster 1 osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552] boot 2023-12-07T17:14:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[25056]: cluster 2023-12-07T17:14:28.691894+0000 mon.a (mon.0) 621 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2023-12-07T17:14:30.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[25056]: audit 2023-12-07T17:14:28.692199+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:30.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[36207]: cluster 2023-12-07T17:14:28.424200+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T17:14:30.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[36207]: cluster 2023-12-07T17:14:28.691826+0000 mon.a (mon.0) 620 : cluster 1 osd.7 [v2:172.21.15.184:6824/648979552,v1:172.21.15.184:6825/648979552] boot 2023-12-07T17:14:30.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[36207]: cluster 2023-12-07T17:14:28.691894+0000 mon.a (mon.0) 621 : cluster 0 osdmap e54: 8 total, 8 up, 8 in 2023-12-07T17:14:30.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:29 smithi064 bash[36207]: audit 2023-12-07T17:14:28.692199+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:14:31.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:30 smithi184 bash[31336]: cluster 2023-12-07T17:14:29.709691+0000 mon.a (mon.0) 623 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2023-12-07T17:14:31.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:30 smithi184 bash[31336]: cluster 2023-12-07T17:14:30.424777+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:31.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:30 smithi064 bash[25056]: cluster 2023-12-07T17:14:29.709691+0000 mon.a (mon.0) 623 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2023-12-07T17:14:31.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:30 smithi064 bash[25056]: cluster 2023-12-07T17:14:30.424777+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:31.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:30 smithi064 bash[36207]: cluster 2023-12-07T17:14:29.709691+0000 mon.a (mon.0) 623 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2023-12-07T17:14:31.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:30 smithi064 bash[36207]: cluster 2023-12-07T17:14:30.424777+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:32.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:31 smithi184 bash[31336]: cluster 2023-12-07T17:14:30.724275+0000 mon.a (mon.0) 624 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2023-12-07T17:14:32.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:31 smithi064 bash[25056]: cluster 2023-12-07T17:14:30.724275+0000 mon.a (mon.0) 624 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2023-12-07T17:14:32.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:31 smithi064 bash[36207]: cluster 2023-12-07T17:14:30.724275+0000 mon.a (mon.0) 624 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2023-12-07T17:14:33.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:32 smithi184 bash[31336]: cluster 2023-12-07T17:14:31.724376+0000 mon.a (mon.0) 625 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2023-12-07T17:14:33.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:32 smithi184 bash[31336]: cluster 2023-12-07T17:14:32.425334+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:32 smithi064 bash[25056]: cluster 2023-12-07T17:14:31.724376+0000 mon.a (mon.0) 625 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2023-12-07T17:14:33.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:32 smithi064 bash[25056]: cluster 2023-12-07T17:14:32.425334+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:33.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:32 smithi064 bash[36207]: cluster 2023-12-07T17:14:31.724376+0000 mon.a (mon.0) 625 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2023-12-07T17:14:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:32 smithi064 bash[36207]: cluster 2023-12-07T17:14:32.425334+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:33.447 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:14:33.999 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:33 smithi184 bash[31336]: cluster 2023-12-07T17:14:32.736050+0000 mon.a (mon.0) 626 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2023-12-07T17:14:34.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:33 smithi064 bash[25056]: cluster 2023-12-07T17:14:32.736050+0000 mon.a (mon.0) 626 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2023-12-07T17:14:34.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:33 smithi064 bash[36207]: cluster 2023-12-07T17:14:32.736050+0000 mon.a (mon.0) 626 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2023-12-07T17:14:35.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:34 smithi184 bash[31336]: cluster 2023-12-07T17:14:34.425895+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-12-07T17:14:35.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:34 smithi064 bash[25056]: cluster 2023-12-07T17:14:34.425895+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-12-07T17:14:35.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:34 smithi064 bash[36207]: cluster 2023-12-07T17:14:34.425895+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-12-07T17:14:36.800 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:14:37.596 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1701969268,"num_in_osds":8,"osd_in_since":1701969240,"num_remapped_pgs":0} 2023-12-07T17:14:37.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T17:14:37.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:37 smithi064 bash[36207]: cluster 2023-12-07T17:14:36.426520+0000 mgr.y (mgr.14152) 336 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T17:14:37.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:37 smithi064 bash[36207]: audit 2023-12-07T17:14:36.798660+0000 mon.a (mon.0) 627 : audit 0 from='client.? 172.21.15.64:0/3406373011' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T17:14:37.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:37 smithi064 bash[25056]: cluster 2023-12-07T17:14:36.426520+0000 mgr.y (mgr.14152) 336 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T17:14:37.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:37 smithi064 bash[25056]: audit 2023-12-07T17:14:36.798660+0000 mon.a (mon.0) 627 : audit 0 from='client.? 172.21.15.64:0/3406373011' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T17:14:38.070 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:37 smithi184 bash[31336]: cluster 2023-12-07T17:14:36.426520+0000 mgr.y (mgr.14152) 336 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T17:14:38.071 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:37 smithi184 bash[31336]: audit 2023-12-07T17:14:36.798660+0000 mon.a (mon.0) 627 : audit 0 from='client.? 172.21.15.64:0/3406373011' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: cluster 2023-12-07T17:14:38.427170+0000 mgr.y (mgr.14152) 337 : cluster 0 pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.494652+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.502260+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.504359+0000 mon.a (mon.0) 630 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.505937+0000 mon.a (mon.0) 631 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.507359+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.508785+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.514528+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.517208+0000 mon.a (mon.0) 635 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:40.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.518347+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:40.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:39 smithi184 bash[31336]: audit 2023-12-07T17:14:39.524859+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: cluster 2023-12-07T17:14:38.427170+0000 mgr.y (mgr.14152) 337 : cluster 0 pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.494652+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.502260+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.504359+0000 mon.a (mon.0) 630 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.505937+0000 mon.a (mon.0) 631 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.507359+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.508785+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.514528+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.517208+0000 mon.a (mon.0) 635 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.518347+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[25056]: audit 2023-12-07T17:14:39.524859+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: cluster 2023-12-07T17:14:38.427170+0000 mgr.y (mgr.14152) 337 : cluster 0 pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.494652+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.502260+0000 mon.a (mon.0) 629 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.504359+0000 mon.a (mon.0) 630 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.505937+0000 mon.a (mon.0) 631 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.507359+0000 mon.a (mon.0) 632 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.508785+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.514528+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.517208+0000 mon.a (mon.0) 635 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.518347+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:14:40.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:39 smithi064 bash[36207]: audit 2023-12-07T17:14:39.524859+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:14:41.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:40 smithi184 bash[31336]: cephadm 2023-12-07T17:14:39.484671+0000 mgr.y (mgr.14152) 338 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:14:41.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:40 smithi184 bash[31336]: cephadm 2023-12-07T17:14:39.509870+0000 mgr.y (mgr.14152) 339 : cephadm 1 Adjusting osd_memory_target on smithi184 to 4316M 2023-12-07T17:14:41.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:40 smithi064 bash[25056]: cephadm 2023-12-07T17:14:39.484671+0000 mgr.y (mgr.14152) 338 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:14:41.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:40 smithi064 bash[25056]: cephadm 2023-12-07T17:14:39.509870+0000 mgr.y (mgr.14152) 339 : cephadm 1 Adjusting osd_memory_target on smithi184 to 4316M 2023-12-07T17:14:41.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:40 smithi064 bash[36207]: cephadm 2023-12-07T17:14:39.484671+0000 mgr.y (mgr.14152) 338 : cephadm 1 Detected new or changed devices on smithi184 2023-12-07T17:14:41.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:40 smithi064 bash[36207]: cephadm 2023-12-07T17:14:39.509870+0000 mgr.y (mgr.14152) 339 : cephadm 1 Adjusting osd_memory_target on smithi184 to 4316M 2023-12-07T17:14:42.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:41 smithi184 bash[31336]: cluster 2023-12-07T17:14:40.427950+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-07T17:14:42.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:41 smithi064 bash[25056]: cluster 2023-12-07T17:14:40.427950+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-07T17:14:42.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:41 smithi064 bash[36207]: cluster 2023-12-07T17:14:40.427950+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-07T17:14:42.365 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:14:44.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:43 smithi184 bash[31336]: cluster 2023-12-07T17:14:42.428554+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T17:14:44.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:43 smithi064 bash[25056]: cluster 2023-12-07T17:14:42.428554+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T17:14:44.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:43 smithi064 bash[36207]: cluster 2023-12-07T17:14:42.428554+0000 mgr.y (mgr.14152) 341 : cluster 0 pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T17:14:45.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:44 smithi184 bash[31336]: cluster 2023-12-07T17:14:44.429148+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-07T17:14:45.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:44 smithi064 bash[25056]: cluster 2023-12-07T17:14:44.429148+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-07T17:14:45.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:44 smithi064 bash[36207]: cluster 2023-12-07T17:14:44.429148+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-07T17:14:45.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:14:45.677 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":58,"fsid":"7ae18f76-9522-11ee-95a2-87774f69a715","created":"2023-12-07T17:04:13.885658+0000","modified":"2023-12-07T17:14:32.727983+0000","last_up_change":"2023-12-07T17:14:28.682548+0000","last_in_change":"2023-12-07T17:14:00.148477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T17:09:56.938253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beb84ad8-fb61-45c8-99fa-eb8a522bcca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6803","nonce":4255677702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6805","nonce":4255677702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6809","nonce":4255677702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6807","nonce":4255677702}]},"public_addr":"172.21.15.64:6803/4255677702","cluster_addr":"172.21.15.64:6805/4255677702","heartbeat_back_addr":"172.21.15.64:6809/4255677702","heartbeat_front_addr":"172.21.15.64:6807/4255677702","state":["exists","up"]},{"osd":1,"uuid":"5fda1ac8-8149-447f-ab1f-5c6192928f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6811","nonce":1129839528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6813","nonce":1129839528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6817","nonce":1129839528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6815","nonce":1129839528}]},"public_addr":"172.21.15.64:6811/1129839528","cluster_addr":"172.21.15.64:6813/1129839528","heartbeat_back_addr":"172.21.15.64:6817/1129839528","heartbeat_front_addr":"172.21.15.64:6815/1129839528","state":["exists","up"]},{"osd":2,"uuid":"36cb5cff-a8e8-46cb-86c5-55243883da50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6819","nonce":135846840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6821","nonce":135846840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6825","nonce":135846840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6823","nonce":135846840}]},"public_addr":"172.21.15.64:6819/135846840","cluster_addr":"172.21.15.64:6821/135846840","heartbeat_back_addr":"172.21.15.64:6825/135846840","heartbeat_front_addr":"172.21.15.64:6823/135846840","state":["exists","up"]},{"osd":3,"uuid":"22c06021-9405-4ee7-8b82-1b3ee1a7b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6827","nonce":2613043191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6829","nonce":2613043191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6833","nonce":2613043191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6831","nonce":2613043191}]},"public_addr":"172.21.15.64:6827/2613043191","cluster_addr":"172.21.15.64:6829/2613043191","heartbeat_back_addr":"172.21.15.64:6833/2613043191","heartbeat_front_addr":"172.21.15.64:6831/2613043191","state":["exists","up"]},{"osd":4,"uuid":"9be6b07a-5939-4ac2-abaa-f889afc87cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6801","nonce":2660335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6803","nonce":2660335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6807","nonce":2660335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6805","nonce":2660335559}]},"public_addr":"172.21.15.184:6801/2660335559","cluster_addr":"172.21.15.184:6803/2660335559","heartbeat_back_addr":"172.21.15.184:6807/2660335559","heartbeat_front_addr":"172.21.15.184:6805/2660335559","state":["exists","up"]},{"osd":5,"uuid":"2df3256b-b1c8-4b26-82c6-55d945e2506c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6809","nonce":911723058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6811","nonce":911723058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6815","nonce":911723058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6813","nonce":911723058}]},"public_addr":"172.21.15.184:6809/911723058","cluster_addr":"172.21.15.184:6811/911723058","heartbeat_back_addr":"172.21.15.184:6815/911723058","heartbeat_front_addr":"172.21.15.184:6813/911723058","state":["exists","up"]},{"osd":6,"uuid":"0ae74eb7-8166-414d-abc5-7f0ecb5f4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6817","nonce":3929847913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6819","nonce":3929847913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6823","nonce":3929847913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6821","nonce":3929847913}]},"public_addr":"172.21.15.184:6817/3929847913","cluster_addr":"172.21.15.184:6819/3929847913","heartbeat_back_addr":"172.21.15.184:6823/3929847913","heartbeat_front_addr":"172.21.15.184:6821/3929847913","state":["exists","up"]},{"osd":7,"uuid":"a1498aa6-78dd-4a46-9a67-4454cb6f66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6825","nonce":648979552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6827","nonce":648979552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6831","nonce":648979552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6829","nonce":648979552}]},"public_addr":"172.21.15.184:6825/648979552","cluster_addr":"172.21.15.184:6827/648979552","heartbeat_back_addr":"172.21.15.184:6831/648979552","heartbeat_front_addr":"172.21.15.184:6829/648979552","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:07:59.780610+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:08:55.703708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:09:53.615215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:10:50.468115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:11:43.718719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:12:37.349365+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:13:32.110469+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:14:26.660719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/2213461460":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/3925146267":"2023-12-08T17:05:26.278541+0000","172.21.15.64:6801/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:6800/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/587612034":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/43744120":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/1799325112":"2023-12-08T17:04:48.222204+0000","172.21.15.64:6800/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:6801/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/2277610507":"2023-12-08T17:05:26.278541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T17:14:46.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:45 smithi184 bash[31336]: audit 2023-12-07T17:14:45.673747+0000 mon.a (mon.0) 638 : audit 0 from='client.? 172.21.15.64:0/541146642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:14:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:45 smithi064 bash[25056]: audit 2023-12-07T17:14:45.673747+0000 mon.a (mon.0) 638 : audit 0 from='client.? 172.21.15.64:0/541146642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:14:46.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:45 smithi064 bash[36207]: audit 2023-12-07T17:14:45.673747+0000 mon.a (mon.0) 638 : audit 0 from='client.? 172.21.15.64:0/541146642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:14:46.524 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T17:09:56.938253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-07T17:14:46.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-07T17:14:46.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:46 smithi064 bash[25056]: cluster 2023-12-07T17:14:46.429735+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:46.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:46 smithi064 bash[36207]: cluster 2023-12-07T17:14:46.429735+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:47.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:46 smithi184 bash[31336]: cluster 2023-12-07T17:14:46.429735+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:49.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:49 smithi184 bash[31336]: cluster 2023-12-07T17:14:48.430332+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:49.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:49 smithi064 bash[25056]: cluster 2023-12-07T17:14:48.430332+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:49.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:49 smithi064 bash[36207]: cluster 2023-12-07T17:14:48.430332+0000 mgr.y (mgr.14152) 344 : cluster 0 pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:51.286 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:14:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:51 smithi184 bash[31336]: cluster 2023-12-07T17:14:50.430944+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:52.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:51 smithi064 bash[25056]: cluster 2023-12-07T17:14:50.430944+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:51 smithi064 bash[36207]: cluster 2023-12-07T17:14:50.430944+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:54.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:53 smithi064 bash[25056]: cluster 2023-12-07T17:14:52.431418+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:54.005 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:53 smithi064 bash[36207]: cluster 2023-12-07T17:14:52.431418+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:54.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:53 smithi184 bash[31336]: cluster 2023-12-07T17:14:52.431418+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:54.493 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-12-07T17:14:54.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:54 smithi064 bash[25056]: audit 2023-12-07T17:14:54.491559+0000 mon.c (mon.2) 11 : audit 0 from='client.? 172.21.15.64:0/4130982298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T17:14:54.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:54 smithi064 bash[36207]: audit 2023-12-07T17:14:54.491559+0000 mon.c (mon.2) 11 : audit 0 from='client.? 172.21.15.64:0/4130982298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T17:14:55.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:54 smithi184 bash[31336]: audit 2023-12-07T17:14:54.491559+0000 mon.c (mon.2) 11 : audit 0 from='client.? 172.21.15.64:0/4130982298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T17:14:55.318 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi064 2023-12-07T17:14:55.318 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi064=foo.a' 2023-12-07T17:14:56.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:55 smithi184 bash[31336]: cluster 2023-12-07T17:14:54.431961+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:56.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:55 smithi064 bash[25056]: cluster 2023-12-07T17:14:54.431961+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:56.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:55 smithi064 bash[36207]: cluster 2023-12-07T17:14:54.431961+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:57.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:56 smithi184 bash[31336]: cluster 2023-12-07T17:14:56.432489+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:57.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:56 smithi064 bash[25056]: cluster 2023-12-07T17:14:56.432489+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:57.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:56 smithi064 bash[36207]: cluster 2023-12-07T17:14:56.432489+0000 mgr.y (mgr.14152) 348 : cluster 0 pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:59.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:14:59 smithi184 bash[31336]: cluster 2023-12-07T17:14:58.433056+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:59.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:14:59 smithi064 bash[25056]: cluster 2023-12-07T17:14:58.433056+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:14:59.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:14:59 smithi064 bash[36207]: cluster 2023-12-07T17:14:58.433056+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:00.073 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:01.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:01 smithi064 bash[25056]: cluster 2023-12-07T17:15:00.433752+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:01.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:01 smithi064 bash[36207]: cluster 2023-12-07T17:15:00.433752+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:01.985 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:01 smithi184 bash[31336]: cluster 2023-12-07T17:15:00.433752+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:03.287 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled rgw.foo.a update... 2023-12-07T17:15:04.038 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:03 smithi184 bash[31336]: cluster 2023-12-07T17:15:02.434350+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:04.038 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:03 smithi184 bash[31336]: audit 2023-12-07T17:15:03.284365+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.038 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:03 smithi184 bash[31336]: audit 2023-12-07T17:15:03.321376+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:04.040 DEBUG:teuthology.orchestra.run.smithi064:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service 2023-12-07T17:15:04.043 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi184 2023-12-07T17:15:04.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2023-12-07T17:15:04.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[25056]: cluster 2023-12-07T17:15:02.434350+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:04.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[25056]: audit 2023-12-07T17:15:03.284365+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.050 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[25056]: audit 2023-12-07T17:15:03.321376+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:04.050 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[36207]: cluster 2023-12-07T17:15:02.434350+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:04.051 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[36207]: audit 2023-12-07T17:15:03.284365+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.051 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:03 smithi064 bash[36207]: audit 2023-12-07T17:15:03.321376+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:04.355 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:15:04.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:03.273469+0000 mgr.y (mgr.14152) 352 : audit 0 from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi064=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:04.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: cephadm 2023-12-07T17:15:03.277386+0000 mgr.y (mgr.14152) 353 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:04.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.009529+0000 mon.a (mon.0) 641 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:04.869 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.011406+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.024257+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.029072+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.033803+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.043574+0000 mon.a (mon.0) 646 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[36207]: audit 2023-12-07T17:15:04.046276+0000 mon.a (mon.0) 647 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:03.273469+0000 mgr.y (mgr.14152) 352 : audit 0 from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi064=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: cephadm 2023-12-07T17:15:03.277386+0000 mgr.y (mgr.14152) 353 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.009529+0000 mon.a (mon.0) 641 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.011406+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.024257+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.029072+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-12-07T17:15:04.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.033803+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-12-07T17:15:04.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.043574+0000 mon.a (mon.0) 646 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:04.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:04 smithi064 bash[25056]: audit 2023-12-07T17:15:04.046276+0000 mon.a (mon.0) 647 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:03.273469+0000 mgr.y (mgr.14152) 352 : audit 0 from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi064=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: cephadm 2023-12-07T17:15:03.277386+0000 mgr.y (mgr.14152) 353 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.009529+0000 mon.a (mon.0) 641 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.011406+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.024257+0000 mon.a (mon.0) 643 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.029072+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.033803+0000 mon.a (mon.0) 645 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.043574+0000 mon.a (mon.0) 646 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:05.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:04 smithi184 bash[31336]: audit 2023-12-07T17:15:04.046276+0000 mon.a (mon.0) 647 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:05.982 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:05 smithi064 bash[25056]: cephadm 2023-12-07T17:15:04.047632+0000 mgr.y (mgr.14152) 354 : cephadm 1 Deploying daemon rgw.foo.a on smithi064 2023-12-07T17:15:05.983 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:05 smithi064 bash[25056]: cluster 2023-12-07T17:15:04.435092+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:05.983 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:05 smithi064 bash[36207]: cephadm 2023-12-07T17:15:04.047632+0000 mgr.y (mgr.14152) 354 : cephadm 1 Deploying daemon rgw.foo.a on smithi064 2023-12-07T17:15:05.983 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:05 smithi064 bash[36207]: cluster 2023-12-07T17:15:04.435092+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:06.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:05 smithi184 bash[31336]: cephadm 2023-12-07T17:15:04.047632+0000 mgr.y (mgr.14152) 354 : cephadm 1 Deploying daemon rgw.foo.a on smithi064 2023-12-07T17:15:06.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:05 smithi184 bash[31336]: cluster 2023-12-07T17:15:04.435092+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:07.094 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:07 smithi064 systemd[1]: Started Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:08.044 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: cluster 2023-12-07T17:15:06.435713+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:08.044 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.079714+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.044 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.087762+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.124618+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.130154+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.133712+0000 mon.a (mon.0) 652 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[25056]: audit 2023-12-07T17:15:07.198606+0000 mon.a (mon.0) 653 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: cluster 2023-12-07T17:15:06.435713+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.079714+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.087762+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.124618+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.130154+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.133712+0000 mon.a (mon.0) 652 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.045 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:07 smithi064 bash[36207]: audit 2023-12-07T17:15:07.198606+0000 mon.a (mon.0) 653 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:08.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: cluster 2023-12-07T17:15:06.435713+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:08.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.079714+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.087762+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.124618+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.130154+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.133712+0000 mon.a (mon.0) 652 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:08.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:07 smithi184 bash[31336]: audit 2023-12-07T17:15:07.198606+0000 mon.a (mon.0) 653 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:08.797 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:09.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:08 smithi184 bash[31336]: cephadm 2023-12-07T17:15:07.125799+0000 mgr.y (mgr.14152) 357 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:09.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:08 smithi184 bash[31336]: cluster 2023-12-07T17:15:08.436272+0000 mgr.y (mgr.14152) 358 : cluster 0 pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:09.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:08 smithi064 bash[25056]: cephadm 2023-12-07T17:15:07.125799+0000 mgr.y (mgr.14152) 357 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:09.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:08 smithi064 bash[25056]: cluster 2023-12-07T17:15:08.436272+0000 mgr.y (mgr.14152) 358 : cluster 0 pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:09.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:08 smithi064 bash[36207]: cephadm 2023-12-07T17:15:07.125799+0000 mgr.y (mgr.14152) 357 : cephadm 1 Saving service rgw.foo.a spec with placement smithi064=foo.a;count:1 2023-12-07T17:15:09.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:08 smithi064 bash[36207]: cluster 2023-12-07T17:15:08.436272+0000 mgr.y (mgr.14152) 358 : cluster 0 pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:10.015 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:09 smithi184 bash[31336]: cluster 2023-12-07T17:15:08.716208+0000 mon.a (mon.0) 654 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2023-12-07T17:15:10.015 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:09 smithi184 bash[31336]: audit 2023-12-07T17:15:08.718533+0000 mon.a (mon.0) 655 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-12-07T17:15:10.015 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:09 smithi184 bash[31336]: audit 2023-12-07T17:15:09.285204+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:10.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[25056]: cluster 2023-12-07T17:15:08.716208+0000 mon.a (mon.0) 654 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2023-12-07T17:15:10.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[25056]: audit 2023-12-07T17:15:08.718533+0000 mon.a (mon.0) 655 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-12-07T17:15:10.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[25056]: audit 2023-12-07T17:15:09.285204+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:10.043 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[36207]: cluster 2023-12-07T17:15:08.716208+0000 mon.a (mon.0) 654 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2023-12-07T17:15:10.043 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[36207]: audit 2023-12-07T17:15:08.718533+0000 mon.a (mon.0) 655 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2023-12-07T17:15:10.043 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[36207]: audit 2023-12-07T17:15:09.285204+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:10.043 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug 2023-12-07T17:15:09.804+0000 7fd84c2b17c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fd84c2b17c0 time 2023-12-07T17:15:09.811374+0000 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd84a0feee7] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: main() 2023-12-07T17:15:10.044 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: __libc_start_main() 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: _start() 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug 2023-12-07T17:15:09.812+0000 7fd84c2b17c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fd84c2b17c0 time 2023-12-07T17:15:09.811374+0000 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd84a0feee7] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: main() 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: __libc_start_main() 2023-12-07T17:15:10.045 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: _start() 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: *** Caught signal (Aborted) ** 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: in thread 7fd84c2b17c0 thread_name:radosgw 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd84746acf0] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: gsignal() 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: abort() 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd84a0fef41] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.046 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 12: main() 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 13: __libc_start_main() 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 14: _start() 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug 2023-12-07T17:15:09.816+0000 7fd84c2b17c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: in thread 7fd84c2b17c0 thread_name:radosgw 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd84746acf0] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: gsignal() 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: abort() 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd84a0fef41] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.047 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 12: main() 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 13: __libc_start_main() 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 14: _start() 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug -6> 2023-12-07T17:15:09.804+0000 7fd84c2b17c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug -1> 2023-12-07T17:15:09.812+0000 7fd84c2b17c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fd84c2b17c0 time 2023-12-07T17:15:09.811374+0000 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd84a0feee7] 2023-12-07T17:15:10.048 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: main() 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: __libc_start_main() 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: _start() 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug 0> 2023-12-07T17:15:09.816+0000 7fd84c2b17c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: in thread 7fd84c2b17c0 thread_name:radosgw 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd84746acf0] 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: gsignal() 2023-12-07T17:15:10.049 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: abort() 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd84a0fef41] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 12: main() 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 13: __libc_start_main() 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 14: _start() 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug -6> 2023-12-07T17:15:09.804+0000 7fd84c2b17c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug -1> 2023-12-07T17:15:09.812+0000 7fd84c2b17c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fd84c2b17c0 time 2023-12-07T17:15:09.811374+0000 2023-12-07T17:15:10.050 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd84a0feee7] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: main() 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: __libc_start_main() 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: _start() 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: debug 0> 2023-12-07T17:15:09.816+0000 7fd84c2b17c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: in thread 7fd84c2b17c0 thread_name:radosgw 2023-12-07T17:15:10.051 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fd84746acf0] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 2: gsignal() 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 3: abort() 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd84a0fef41] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fd84a0ff0ad] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fd84a1edf69] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x5580b8fa991d] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 8: (RGWRados::init_rados()+0x17a) [0x5580b92411fa] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x5580b9251ba0] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5580b91370ee] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 11: (rgw::AppMain::init_storage()+0x350) [0x5580b8ce9680] 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 12: main() 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 13: __libc_start_main() 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: 14: _start() 2023-12-07T17:15:10.052 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:09 smithi064 bash[85326]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:10.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:10 smithi184 bash[31336]: audit 2023-12-07T17:15:09.714235+0000 mon.a (mon.0) 657 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-12-07T17:15:10.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:10 smithi184 bash[31336]: cluster 2023-12-07T17:15:09.720930+0000 mon.a (mon.0) 658 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2023-12-07T17:15:10.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:10 smithi184 bash[31336]: cluster 2023-12-07T17:15:10.436903+0000 mgr.y (mgr.14152) 359 : cluster 0 pgmap v342: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:11.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[25056]: audit 2023-12-07T17:15:09.714235+0000 mon.a (mon.0) 657 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-12-07T17:15:11.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[25056]: cluster 2023-12-07T17:15:09.720930+0000 mon.a (mon.0) 658 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2023-12-07T17:15:11.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[25056]: cluster 2023-12-07T17:15:10.436903+0000 mgr.y (mgr.14152) 359 : cluster 0 pgmap v342: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:11.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[36207]: audit 2023-12-07T17:15:09.714235+0000 mon.a (mon.0) 657 : audit 1 from='client.? 172.21.15.64:0/3251863344' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2023-12-07T17:15:11.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[36207]: cluster 2023-12-07T17:15:09.720930+0000 mon.a (mon.0) 658 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2023-12-07T17:15:11.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:10 smithi064 bash[36207]: cluster 2023-12-07T17:15:10.436903+0000 mgr.y (mgr.14152) 359 : cluster 0 pgmap v342: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:11.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:10 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=134/n/a 2023-12-07T17:15:11.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:10 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:15:12.210 INFO:teuthology.orchestra.run.smithi184.stderr:pool 'datapool' created 2023-12-07T17:15:12.473 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:12 smithi184 bash[31336]: cluster 2023-12-07T17:15:10.739452+0000 mon.a (mon.0) 659 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2023-12-07T17:15:12.473 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:12 smithi184 bash[31336]: audit 2023-12-07T17:15:11.694878+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.184:0/2632524303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:12.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:12 smithi064 bash[25056]: cluster 2023-12-07T17:15:10.739452+0000 mon.a (mon.0) 659 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2023-12-07T17:15:12.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:12 smithi064 bash[25056]: audit 2023-12-07T17:15:11.694878+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.184:0/2632524303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:12.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:12 smithi064 bash[36207]: cluster 2023-12-07T17:15:10.739452+0000 mon.a (mon.0) 659 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2023-12-07T17:15:12.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:12 smithi064 bash[36207]: audit 2023-12-07T17:15:11.694878+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.184:0/2632524303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:12.904 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- rbd pool init datapool 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:11.695919+0000 mon.a (mon.0) 660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:12.198244+0000 mon.a (mon.0) 661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: cluster 2023-12-07T17:15:12.202279+0000 mon.a (mon.0) 662 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: cluster 2023-12-07T17:15:12.437954+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v345: 36 pgs: 5 creating+peering, 6 active+clean, 25 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:12.841930+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:12.853141+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:12.855226+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:13.154 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:13 smithi184 bash[31336]: audit 2023-12-07T17:15:12.856707+0000 mon.a (mon.0) 666 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:13.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:11.695919+0000 mon.a (mon.0) 660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:13.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:12.198244+0000 mon.a (mon.0) 661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2023-12-07T17:15:13.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: cluster 2023-12-07T17:15:12.202279+0000 mon.a (mon.0) 662 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2023-12-07T17:15:13.619 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: cluster 2023-12-07T17:15:12.437954+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v345: 36 pgs: 5 creating+peering, 6 active+clean, 25 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:12.841930+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:12.853141+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:12.855226+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[36207]: audit 2023-12-07T17:15:12.856707+0000 mon.a (mon.0) 666 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:11.695919+0000 mon.a (mon.0) 660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:12.198244+0000 mon.a (mon.0) 661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: cluster 2023-12-07T17:15:12.202279+0000 mon.a (mon.0) 662 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: cluster 2023-12-07T17:15:12.437954+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v345: 36 pgs: 5 creating+peering, 6 active+clean, 25 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:12.841930+0000 mon.a (mon.0) 663 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:12.853141+0000 mon.a (mon.0) 664 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:12.855226+0000 mon.a (mon.0) 665 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:13 smithi064 bash[25056]: audit 2023-12-07T17:15:12.856707+0000 mon.a (mon.0) 666 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:14.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: cluster 2023-12-07T17:15:12.859417+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v346: 36 pgs: 6 creating+peering, 8 active+clean, 22 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 493 B/s wr, 0 op/s 2023-12-07T17:15:14.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: cephadm 2023-12-07T17:15:12.865226+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: cluster 2023-12-07T17:15:13.198165+0000 mon.a (mon.0) 667 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: cluster 2023-12-07T17:15:13.198219+0000 mon.a (mon.0) 668 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: cluster 2023-12-07T17:15:13.212755+0000 mon.a (mon.0) 669 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.275333+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.356476+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.885458+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.893247+0000 mon.a (mon.0) 673 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.895174+0000 mon.a (mon.0) 674 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:14 smithi184 bash[31336]: audit 2023-12-07T17:15:13.896615+0000 mon.a (mon.0) 675 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:14.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: cluster 2023-12-07T17:15:12.859417+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v346: 36 pgs: 6 creating+peering, 8 active+clean, 22 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 493 B/s wr, 0 op/s 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: cephadm 2023-12-07T17:15:12.865226+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: cluster 2023-12-07T17:15:13.198165+0000 mon.a (mon.0) 667 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: cluster 2023-12-07T17:15:13.198219+0000 mon.a (mon.0) 668 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: cluster 2023-12-07T17:15:13.212755+0000 mon.a (mon.0) 669 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.275333+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.356476+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.885458+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.893247+0000 mon.a (mon.0) 673 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.895174+0000 mon.a (mon.0) 674 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[25056]: audit 2023-12-07T17:15:13.896615+0000 mon.a (mon.0) 675 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: cluster 2023-12-07T17:15:12.859417+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v346: 36 pgs: 6 creating+peering, 8 active+clean, 22 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 493 B/s wr, 0 op/s 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: cephadm 2023-12-07T17:15:12.865226+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: cluster 2023-12-07T17:15:13.198165+0000 mon.a (mon.0) 667 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: cluster 2023-12-07T17:15:13.198219+0000 mon.a (mon.0) 668 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: cluster 2023-12-07T17:15:13.212755+0000 mon.a (mon.0) 669 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.275333+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.356476+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.885458+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.893247+0000 mon.a (mon.0) 673 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.895174+0000 mon.a (mon.0) 674 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:14.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:14 smithi064 bash[36207]: audit 2023-12-07T17:15:13.896615+0000 mon.a (mon.0) 675 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:15 smithi184 bash[31336]: cluster 2023-12-07T17:15:13.899537+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v348: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-07T17:15:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:15 smithi184 bash[31336]: cluster 2023-12-07T17:15:13.899955+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v349: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-12-07T17:15:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:15 smithi184 bash[31336]: cephadm 2023-12-07T17:15:13.904734+0000 mgr.y (mgr.14152) 365 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:15 smithi184 bash[31336]: cluster 2023-12-07T17:15:14.215703+0000 mon.a (mon.0) 676 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2023-12-07T17:15:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:15 smithi184 bash[31336]: audit 2023-12-07T17:15:14.296389+0000 mon.a (mon.0) 677 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:15.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[25056]: cluster 2023-12-07T17:15:13.899537+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v348: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-07T17:15:15.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[25056]: cluster 2023-12-07T17:15:13.899955+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v349: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[25056]: cephadm 2023-12-07T17:15:13.904734+0000 mgr.y (mgr.14152) 365 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[25056]: cluster 2023-12-07T17:15:14.215703+0000 mon.a (mon.0) 676 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[25056]: audit 2023-12-07T17:15:14.296389+0000 mon.a (mon.0) 677 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[36207]: cluster 2023-12-07T17:15:13.899537+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v348: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[36207]: cluster 2023-12-07T17:15:13.899955+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v349: 36 pgs: 8 creating+peering, 28 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[36207]: cephadm 2023-12-07T17:15:13.904734+0000 mgr.y (mgr.14152) 365 : cephadm 1 Checking dashboard <-> RGW credentials 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[36207]: cluster 2023-12-07T17:15:14.215703+0000 mon.a (mon.0) 676 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2023-12-07T17:15:15.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:15 smithi064 bash[36207]: audit 2023-12-07T17:15:14.296389+0000 mon.a (mon.0) 677 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:17.655 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:17 smithi184 bash[31336]: cluster 2023-12-07T17:15:15.900770+0000 mgr.y (mgr.14152) 366 : cluster 0 pgmap v351: 36 pgs: 3 creating+peering, 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-07T17:15:18.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:17 smithi064 bash[25056]: cluster 2023-12-07T17:15:15.900770+0000 mgr.y (mgr.14152) 366 : cluster 0 pgmap v351: 36 pgs: 3 creating+peering, 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-07T17:15:18.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:17 smithi064 bash[36207]: cluster 2023-12-07T17:15:15.900770+0000 mgr.y (mgr.14152) 366 : cluster 0 pgmap v351: 36 pgs: 3 creating+peering, 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2023-12-07T17:15:19.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:19 smithi184 bash[31336]: cluster 2023-12-07T17:15:17.901651+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v352: 36 pgs: 2 creating+peering, 34 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 812 B/s wr, 1 op/s 2023-12-07T17:15:20.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:19 smithi064 bash[25056]: cluster 2023-12-07T17:15:17.901651+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v352: 36 pgs: 2 creating+peering, 34 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 812 B/s wr, 1 op/s 2023-12-07T17:15:20.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:19 smithi064 bash[36207]: cluster 2023-12-07T17:15:17.901651+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v352: 36 pgs: 2 creating+peering, 34 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 812 B/s wr, 1 op/s 2023-12-07T17:15:21.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:20 smithi184 bash[31336]: audit 2023-12-07T17:15:20.380283+0000 mon.b (mon.1) 27 : audit 1 from='client.? 172.21.15.184:0/1644220013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:20 smithi184 bash[31336]: audit 2023-12-07T17:15:20.380763+0000 mon.a (mon.0) 678 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:20 smithi064 bash[25056]: audit 2023-12-07T17:15:20.380283+0000 mon.b (mon.1) 27 : audit 1 from='client.? 172.21.15.184:0/1644220013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:20 smithi064 bash[25056]: audit 2023-12-07T17:15:20.380763+0000 mon.a (mon.0) 678 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:20 smithi064 bash[36207]: audit 2023-12-07T17:15:20.380283+0000 mon.b (mon.1) 27 : audit 1 from='client.? 172.21.15.184:0/1644220013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:20 smithi064 bash[36207]: audit 2023-12-07T17:15:20.380763+0000 mon.a (mon.0) 678 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2023-12-07T17:15:21.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:20 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Scheduled restart job, restart counter is at 1. 2023-12-07T17:15:21.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:20 smithi064 systemd[1]: Stopped Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:21.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:20 smithi064 systemd[1]: Started Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:22.041 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[25056]: cluster 2023-12-07T17:15:19.902615+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v353: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:22.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[25056]: audit 2023-12-07T17:15:20.707053+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2023-12-07T17:15:22.042 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[25056]: cluster 2023-12-07T17:15:20.710137+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2023-12-07T17:15:22.042 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[36207]: cluster 2023-12-07T17:15:19.902615+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v353: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:22.042 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[36207]: audit 2023-12-07T17:15:20.707053+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2023-12-07T17:15:22.042 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:21 smithi064 bash[36207]: cluster 2023-12-07T17:15:20.710137+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2023-12-07T17:15:22.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:21 smithi184 bash[31336]: cluster 2023-12-07T17:15:19.902615+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v353: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:22.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:21 smithi184 bash[31336]: audit 2023-12-07T17:15:20.707053+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2023-12-07T17:15:22.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:21 smithi184 bash[31336]: cluster 2023-12-07T17:15:20.710137+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug 2023-12-07T17:15:22.256+0000 7fad89b727c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fad89b727c0 time 2023-12-07T17:15:22.261707+0000 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fad879bfee7] 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.329 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: main() 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: __libc_start_main() 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: _start() 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug 2023-12-07T17:15:22.260+0000 7fad89b727c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fad89b727c0 time 2023-12-07T17:15:22.261707+0000 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fad879bfee7] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.330 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: main() 2023-12-07T17:15:22.331 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: __libc_start_main() 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: _start() 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: *** Caught signal (Aborted) ** 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: in thread 7fad89b727c0 thread_name:radosgw 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad84d2bcf0] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: gsignal() 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: abort() 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fad879bff41] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.332 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 12: main() 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 13: __libc_start_main() 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 14: _start() 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug 2023-12-07T17:15:22.264+0000 7fad89b727c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: in thread 7fad89b727c0 thread_name:radosgw 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad84d2bcf0] 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: gsignal() 2023-12-07T17:15:22.333 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: abort() 2023-12-07T17:15:22.334 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fad879bff41] 2023-12-07T17:15:22.334 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.334 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.334 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 12: main() 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 13: __libc_start_main() 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 14: _start() 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug -6> 2023-12-07T17:15:22.256+0000 7fad89b727c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug -1> 2023-12-07T17:15:22.260+0000 7fad89b727c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fad89b727c0 time 2023-12-07T17:15:22.261707+0000 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:22.335 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fad879bfee7] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: main() 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: __libc_start_main() 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: _start() 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug 0> 2023-12-07T17:15:22.264+0000 7fad89b727c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: in thread 7fad89b727c0 thread_name:radosgw 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.336 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad84d2bcf0] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: gsignal() 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: abort() 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fad879bff41] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 12: main() 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 13: __libc_start_main() 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 14: _start() 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:22.337 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug -6> 2023-12-07T17:15:22.256+0000 7fad89b727c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug -1> 2023-12-07T17:15:22.260+0000 7fad89b727c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fad89b727c0 time 2023-12-07T17:15:22.261707+0000 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fad879bfee7] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: main() 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: __libc_start_main() 2023-12-07T17:15:22.338 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: _start() 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: debug 0> 2023-12-07T17:15:22.264+0000 7fad89b727c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: in thread 7fad89b727c0 thread_name:radosgw 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad84d2bcf0] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 2: gsignal() 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 3: abort() 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fad879bff41] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fad879c00ad] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fad87aaef69] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55e6adcde91d] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 8: (RGWRados::init_rados()+0x17a) [0x55e6adf761fa] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55e6adf86ba0] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55e6ade6c0ee] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 11: (rgw::AppMain::init_storage()+0x350) [0x55e6ada1e680] 2023-12-07T17:15:22.339 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 12: main() 2023-12-07T17:15:22.340 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 13: __libc_start_main() 2023-12-07T17:15:22.340 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: 14: _start() 2023-12-07T17:15:22.340 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[89665]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:22.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[25056]: cluster 2023-12-07T17:15:21.714020+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2023-12-07T17:15:22.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[25056]: cluster 2023-12-07T17:15:21.903351+0000 mgr.y (mgr.14152) 369 : cluster 0 pgmap v356: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:22.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[36207]: cluster 2023-12-07T17:15:21.714020+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2023-12-07T17:15:22.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:22 smithi064 bash[36207]: cluster 2023-12-07T17:15:21.903351+0000 mgr.y (mgr.14152) 369 : cluster 0 pgmap v356: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:23.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:22 smithi184 bash[31336]: cluster 2023-12-07T17:15:21.714020+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2023-12-07T17:15:23.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:22 smithi184 bash[31336]: cluster 2023-12-07T17:15:21.903351+0000 mgr.y (mgr.14152) 369 : cluster 0 pgmap v356: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:23.476 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.184 --placement '1;smithi184=iscsi.a' 2023-12-07T17:15:23.619 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:23 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=134/n/a 2023-12-07T17:15:23.619 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:23 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:15:23.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:23 smithi184 bash[31336]: cluster 2023-12-07T17:15:22.710761+0000 mon.a (mon.0) 682 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T17:15:23.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:23 smithi184 bash[31336]: cluster 2023-12-07T17:15:22.724057+0000 mon.a (mon.0) 683 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2023-12-07T17:15:24.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:23 smithi064 bash[25056]: cluster 2023-12-07T17:15:22.710761+0000 mon.a (mon.0) 682 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T17:15:24.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:23 smithi064 bash[25056]: cluster 2023-12-07T17:15:22.724057+0000 mon.a (mon.0) 683 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2023-12-07T17:15:24.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:23 smithi064 bash[36207]: cluster 2023-12-07T17:15:22.710761+0000 mon.a (mon.0) 682 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T17:15:24.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:23 smithi064 bash[36207]: cluster 2023-12-07T17:15:22.724057+0000 mon.a (mon.0) 683 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2023-12-07T17:15:25.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:24 smithi184 bash[31336]: cluster 2023-12-07T17:15:23.904197+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v358: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2023-12-07T17:15:25.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:24 smithi064 bash[25056]: cluster 2023-12-07T17:15:23.904197+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v358: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2023-12-07T17:15:25.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:24 smithi064 bash[36207]: cluster 2023-12-07T17:15:23.904197+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v358: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2023-12-07T17:15:27.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:26 smithi184 bash[31336]: audit 2023-12-07T17:15:26.352220+0000 mon.a (mon.0) 684 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2023-12-07T17:15:27.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:26 smithi064 bash[25056]: audit 2023-12-07T17:15:26.352220+0000 mon.a (mon.0) 684 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2023-12-07T17:15:27.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:26 smithi064 bash[36207]: audit 2023-12-07T17:15:26.352220+0000 mon.a (mon.0) 684 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2023-12-07T17:15:28.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:27 smithi184 bash[31336]: cluster 2023-12-07T17:15:25.904864+0000 mgr.y (mgr.14152) 371 : cluster 0 pgmap v359: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:28.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:27 smithi184 bash[31336]: audit 2023-12-07T17:15:26.702988+0000 mon.a (mon.0) 685 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2023-12-07T17:15:28.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:27 smithi184 bash[31336]: cluster 2023-12-07T17:15:26.709058+0000 mon.a (mon.0) 686 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2023-12-07T17:15:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[25056]: cluster 2023-12-07T17:15:25.904864+0000 mgr.y (mgr.14152) 371 : cluster 0 pgmap v359: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:28.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[25056]: audit 2023-12-07T17:15:26.702988+0000 mon.a (mon.0) 685 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2023-12-07T17:15:28.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[25056]: cluster 2023-12-07T17:15:26.709058+0000 mon.a (mon.0) 686 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2023-12-07T17:15:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[36207]: cluster 2023-12-07T17:15:25.904864+0000 mgr.y (mgr.14152) 371 : cluster 0 pgmap v359: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[36207]: audit 2023-12-07T17:15:26.702988+0000 mon.a (mon.0) 685 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2023-12-07T17:15:28.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:27 smithi064 bash[36207]: cluster 2023-12-07T17:15:26.709058+0000 mon.a (mon.0) 686 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2023-12-07T17:15:28.219 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:28 smithi184 bash[31336]: cluster 2023-12-07T17:15:27.713112+0000 mon.a (mon.0) 687 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2023-12-07T17:15:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:28 smithi184 bash[31336]: cluster 2023-12-07T17:15:27.905635+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v362: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:28 smithi064 bash[25056]: cluster 2023-12-07T17:15:27.713112+0000 mon.a (mon.0) 687 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2023-12-07T17:15:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:28 smithi064 bash[25056]: cluster 2023-12-07T17:15:27.905635+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v362: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:28 smithi064 bash[36207]: cluster 2023-12-07T17:15:27.713112+0000 mon.a (mon.0) 687 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2023-12-07T17:15:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:28 smithi064 bash[36207]: cluster 2023-12-07T17:15:27.905635+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v362: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2023-12-07T17:15:31.224 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled iscsi.datapool update... 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: cluster 2023-12-07T17:15:29.906535+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v363: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 854 B/s rd, 284 B/s wr, 1 op/s 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: cluster 2023-12-07T17:15:30.699277+0000 mon.a (mon.0) 688 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.221673+0000 mon.a (mon.0) 689 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.228826+0000 mon.a (mon.0) 690 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.231801+0000 mon.a (mon.0) 691 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:31.758 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.233326+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:31.759 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.243053+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:31.759 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.248941+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-12-07T17:15:31.759 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.251376+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-12-07T17:15:31.759 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:31 smithi184 bash[31336]: audit 2023-12-07T17:15:31.269216+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: cluster 2023-12-07T17:15:29.906535+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v363: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 854 B/s rd, 284 B/s wr, 1 op/s 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: cluster 2023-12-07T17:15:30.699277+0000 mon.a (mon.0) 688 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.221673+0000 mon.a (mon.0) 689 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.228826+0000 mon.a (mon.0) 690 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.231801+0000 mon.a (mon.0) 691 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.233326+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.243053+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.248941+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.251376+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[25056]: audit 2023-12-07T17:15:31.269216+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: cluster 2023-12-07T17:15:29.906535+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v363: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 854 B/s rd, 284 B/s wr, 1 op/s 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: cluster 2023-12-07T17:15:30.699277+0000 mon.a (mon.0) 688 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.221673+0000 mon.a (mon.0) 689 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.228826+0000 mon.a (mon.0) 690 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.231801+0000 mon.a (mon.0) 691 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.233326+0000 mon.a (mon.0) 692 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.243053+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.248941+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2023-12-07T17:15:32.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.251376+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2023-12-07T17:15:32.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:31 smithi064 bash[36207]: audit 2023-12-07T17:15:31.269216+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:32.254 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2023-12-07T17:15:32.254 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:15:32.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2023-12-07T17:15:32.269 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:15:32.269 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2023-12-07T17:15:32.284 DEBUG:teuthology.orchestra.run.smithi184:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@iscsi.iscsi.a.service 2023-12-07T17:15:32.331 INFO:tasks.cephadm:Adding prometheus.a on smithi184 2023-12-07T17:15:32.332 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply prometheus '1;smithi184=a' 2023-12-07T17:15:32.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:15:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:32 smithi184 bash[31336]: audit 2023-12-07T17:15:31.210288+0000 mgr.y (mgr.14152) 374 : audit 0 from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:32 smithi184 bash[31336]: cephadm 2023-12-07T17:15:31.216008+0000 mgr.y (mgr.14152) 375 : cephadm 1 Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2023-12-07T17:15:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:32 smithi184 bash[31336]: cluster 2023-12-07T17:15:31.236224+0000 mgr.y (mgr.14152) 376 : cluster 0 pgmap v365: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:32 smithi184 bash[31336]: cluster 2023-12-07T17:15:31.236658+0000 mgr.y (mgr.14152) 377 : cluster 0 pgmap v366: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:32 smithi184 bash[31336]: cephadm 2023-12-07T17:15:31.270652+0000 mgr.y (mgr.14152) 378 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi184 2023-12-07T17:15:33.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[36207]: audit 2023-12-07T17:15:31.210288+0000 mgr.y (mgr.14152) 374 : audit 0 from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[36207]: cephadm 2023-12-07T17:15:31.216008+0000 mgr.y (mgr.14152) 375 : cephadm 1 Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[36207]: cluster 2023-12-07T17:15:31.236224+0000 mgr.y (mgr.14152) 376 : cluster 0 pgmap v365: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[36207]: cluster 2023-12-07T17:15:31.236658+0000 mgr.y (mgr.14152) 377 : cluster 0 pgmap v366: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[36207]: cephadm 2023-12-07T17:15:31.270652+0000 mgr.y (mgr.14152) 378 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi184 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[25056]: audit 2023-12-07T17:15:31.210288+0000 mgr.y (mgr.14152) 374 : audit 0 from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[25056]: cephadm 2023-12-07T17:15:31.216008+0000 mgr.y (mgr.14152) 375 : cephadm 1 Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[25056]: cluster 2023-12-07T17:15:31.236224+0000 mgr.y (mgr.14152) 376 : cluster 0 pgmap v365: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[25056]: cluster 2023-12-07T17:15:31.236658+0000 mgr.y (mgr.14152) 377 : cluster 0 pgmap v366: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:33.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:32 smithi064 bash[25056]: cephadm 2023-12-07T17:15:31.270652+0000 mgr.y (mgr.14152) 378 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi184 2023-12-07T17:15:33.869 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:33 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Scheduled restart job, restart counter is at 2. 2023-12-07T17:15:33.869 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:33 smithi064 systemd[1]: Stopped Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:33.869 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:33 smithi064 systemd[1]: Started Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:34.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:34 smithi184 systemd[1]: Started Ceph iscsi.iscsi.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: cluster 2023-12-07T17:15:33.237368+0000 mgr.y (mgr.14152) 379 : cluster 0 pgmap v367: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: audit 2023-12-07T17:15:34.315477+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: audit 2023-12-07T17:15:34.323320+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: audit 2023-12-07T17:15:34.352141+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: audit 2023-12-07T17:15:34.369726+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.788 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[36207]: audit 2023-12-07T17:15:34.435876+0000 mon.a (mon.0) 701 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: cluster 2023-12-07T17:15:33.237368+0000 mgr.y (mgr.14152) 379 : cluster 0 pgmap v367: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: audit 2023-12-07T17:15:34.315477+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: audit 2023-12-07T17:15:34.323320+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: audit 2023-12-07T17:15:34.352141+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: audit 2023-12-07T17:15:34.369726+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:34.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[25056]: audit 2023-12-07T17:15:34.435876+0000 mon.a (mon.0) 701 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug 2023-12-07T17:15:34.715+0000 7fcef45087c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fcef45087c0 time 2023-12-07T17:15:34.719964+0000 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcef2355ee7] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.790 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: main() 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: __libc_start_main() 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: _start() 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug 2023-12-07T17:15:34.719+0000 7fcef45087c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fcef45087c0 time 2023-12-07T17:15:34.719964+0000 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcef2355ee7] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.791 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: main() 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: __libc_start_main() 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: _start() 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: *** Caught signal (Aborted) ** 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: in thread 7fcef45087c0 thread_name:radosgw 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fceef6c1cf0] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: gsignal() 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: abort() 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcef2355f41] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.792 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 12: main() 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 13: __libc_start_main() 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 14: _start() 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug 2023-12-07T17:15:34.723+0000 7fcef45087c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: in thread 7fcef45087c0 thread_name:radosgw 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.793 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fceef6c1cf0] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: gsignal() 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: abort() 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcef2355f41] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 12: main() 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 13: __libc_start_main() 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 14: _start() 2023-12-07T17:15:34.794 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug -6> 2023-12-07T17:15:34.715+0000 7fcef45087c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug -1> 2023-12-07T17:15:34.719+0000 7fcef45087c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fcef45087c0 time 2023-12-07T17:15:34.719964+0000 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcef2355ee7] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.795 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: main() 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: __libc_start_main() 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: _start() 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug 0> 2023-12-07T17:15:34.723+0000 7fcef45087c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: in thread 7fcef45087c0 thread_name:radosgw 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fceef6c1cf0] 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: gsignal() 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: abort() 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcef2355f41] 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.796 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 12: main() 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 13: __libc_start_main() 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 14: _start() 2023-12-07T17:15:34.797 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug -6> 2023-12-07T17:15:34.715+0000 7fcef45087c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug -1> 2023-12-07T17:15:34.719+0000 7fcef45087c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fcef45087c0 time 2023-12-07T17:15:34.719964+0000 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcef2355ee7] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.798 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: main() 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: __libc_start_main() 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: _start() 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: debug 0> 2023-12-07T17:15:34.723+0000 7fcef45087c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: in thread 7fcef45087c0 thread_name:radosgw 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fceef6c1cf0] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 2: gsignal() 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 3: abort() 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcef2355f41] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fcef23560ad] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fcef2444f69] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55eed7a2c91d] 2023-12-07T17:15:34.799 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 8: (RGWRados::init_rados()+0x17a) [0x55eed7cc41fa] 2023-12-07T17:15:34.800 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55eed7cd4ba0] 2023-12-07T17:15:34.800 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55eed7bba0ee] 2023-12-07T17:15:34.800 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 11: (rgw::AppMain::init_storage()+0x350) [0x55eed776c680] 2023-12-07T17:15:34.800 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 12: main() 2023-12-07T17:15:34.801 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 13: __libc_start_main() 2023-12-07T17:15:34.801 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: 14: _start() 2023-12-07T17:15:34.801 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:34 smithi064 bash[90363]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:35.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: cluster 2023-12-07T17:15:33.237368+0000 mgr.y (mgr.14152) 379 : cluster 0 pgmap v367: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:35.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: audit 2023-12-07T17:15:34.315477+0000 mon.a (mon.0) 697 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:35.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: audit 2023-12-07T17:15:34.323320+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:35.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: audit 2023-12-07T17:15:34.352141+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:35.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: audit 2023-12-07T17:15:34.369726+0000 mon.a (mon.0) 700 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:35.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:34 smithi184 bash[31336]: audit 2023-12-07T17:15:34.435876+0000 mon.a (mon.0) 701 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:15:35.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:35 smithi064 bash[25056]: cephadm 2023-12-07T17:15:34.353557+0000 mgr.y (mgr.14152) 380 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2023-12-07T17:15:35.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:35 smithi064 bash[36207]: cephadm 2023-12-07T17:15:34.353557+0000 mgr.y (mgr.14152) 380 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2023-12-07T17:15:35.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:35 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=134/n/a 2023-12-07T17:15:35.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:35 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:15:35.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:35 smithi184 bash[31336]: cephadm 2023-12-07T17:15:34.353557+0000 mgr.y (mgr.14152) 380 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2023-12-07T17:15:35.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:35 smithi184 bash[79402]: log file path now is '/var/log/tcmu-runner.log' 2023-12-07T17:15:36.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug Started the configuration object watcher 2023-12-07T17:15:36.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug Checking for config object changes every 1s 2023-12-07T17:15:36.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug Processing osd blocklist entries for this node 2023-12-07T17:15:36.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[31336]: cluster 2023-12-07T17:15:35.238148+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v368: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:36.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[31336]: audit 2023-12-07T17:15:36.512069+0000 mon.b (mon.1) 28 : audit 0 from='client.? 172.21.15.184:0/2946114188' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T17:15:36.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug Reading the configuration object to update local LIO configuration 2023-12-07T17:15:36.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug Configuration does not have an entry for this host(smithi184.front.sepia.ceph.com) - nothing to define to LIO 2023-12-07T17:15:36.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:36 smithi184 bash[79431]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2023-12-07T17:15:37.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:36 smithi064 bash[25056]: cluster 2023-12-07T17:15:35.238148+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v368: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:37.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:36 smithi064 bash[25056]: audit 2023-12-07T17:15:36.512069+0000 mon.b (mon.1) 28 : audit 0 from='client.? 172.21.15.184:0/2946114188' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T17:15:37.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:36 smithi064 bash[36207]: cluster 2023-12-07T17:15:35.238148+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v368: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:15:37.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:36 smithi064 bash[36207]: audit 2023-12-07T17:15:36.512069+0000 mon.b (mon.1) 28 : audit 0 from='client.? 172.21.15.184:0/2946114188' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T17:15:38.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:37 smithi184 bash[31336]: cluster 2023-12-07T17:15:36.725824+0000 mon.a (mon.0) 702 : cluster 0 mgrmap e16: y(active, since 10m), standbys: x 2023-12-07T17:15:38.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:37 smithi184 bash[31336]: cluster 2023-12-07T17:15:37.238913+0000 mgr.y (mgr.14152) 382 : cluster 0 pgmap v369: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 279 B/s rd, 0 op/s 2023-12-07T17:15:38.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:37 smithi064 bash[25056]: cluster 2023-12-07T17:15:36.725824+0000 mon.a (mon.0) 702 : cluster 0 mgrmap e16: y(active, since 10m), standbys: x 2023-12-07T17:15:38.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:37 smithi064 bash[25056]: cluster 2023-12-07T17:15:37.238913+0000 mgr.y (mgr.14152) 382 : cluster 0 pgmap v369: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 279 B/s rd, 0 op/s 2023-12-07T17:15:38.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:37 smithi064 bash[36207]: cluster 2023-12-07T17:15:36.725824+0000 mon.a (mon.0) 702 : cluster 0 mgrmap e16: y(active, since 10m), standbys: x 2023-12-07T17:15:38.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:37 smithi064 bash[36207]: cluster 2023-12-07T17:15:37.238913+0000 mgr.y (mgr.14152) 382 : cluster 0 pgmap v369: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 279 B/s rd, 0 op/s 2023-12-07T17:15:38.122 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:40.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:40 smithi184 bash[31336]: cluster 2023-12-07T17:15:39.239800+0000 mgr.y (mgr.14152) 383 : cluster 0 pgmap v370: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 119 B/s wr, 2 op/s 2023-12-07T17:15:40.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:40 smithi184 bash[31336]: audit 2023-12-07T17:15:39.295005+0000 mon.a (mon.0) 703 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:40.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:40 smithi064 bash[25056]: cluster 2023-12-07T17:15:39.239800+0000 mgr.y (mgr.14152) 383 : cluster 0 pgmap v370: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 119 B/s wr, 2 op/s 2023-12-07T17:15:40.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:40 smithi064 bash[25056]: audit 2023-12-07T17:15:39.295005+0000 mon.a (mon.0) 703 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:40.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:40 smithi064 bash[36207]: cluster 2023-12-07T17:15:39.239800+0000 mgr.y (mgr.14152) 383 : cluster 0 pgmap v370: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 119 B/s wr, 2 op/s 2023-12-07T17:15:40.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:40 smithi064 bash[36207]: audit 2023-12-07T17:15:39.295005+0000 mon.a (mon.0) 703 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:40.975 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled prometheus update... 2023-12-07T17:15:41.658 DEBUG:teuthology.orchestra.run.smithi184:prometheus.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@prometheus.a.service 2023-12-07T17:15:41.660 INFO:tasks.cephadm:Adding node-exporter.a on smithi064 2023-12-07T17:15:41.660 INFO:tasks.cephadm:Adding node-exporter.b on smithi184 2023-12-07T17:15:41.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply node-exporter '2;smithi064=a;smithi184=b' 2023-12-07T17:15:41.790 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:15:42.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:40.960063+0000 mgr.y (mgr.14152) 384 : audit 0 from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:42.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: cephadm 2023-12-07T17:15:40.964114+0000 mgr.y (mgr.14152) 385 : cephadm 1 Saving service prometheus spec with placement smithi184=a;count:1 2023-12-07T17:15:42.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:40.972345+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:41.036453+0000 mon.a (mon.0) 705 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:41.044524+0000 mon.a (mon.0) 706 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:41.046336+0000 mon.a (mon.0) 707 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:41.047654+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: cluster 2023-12-07T17:15:41.050434+0000 mgr.y (mgr.14152) 386 : cluster 0 pgmap v371: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 104 B/s wr, 1 op/s 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: audit 2023-12-07T17:15:41.055997+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:41 smithi184 bash[31336]: cephadm 2023-12-07T17:15:41.322201+0000 mgr.y (mgr.14152) 387 : cephadm 1 Deploying daemon prometheus.a on smithi184 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:40.960063+0000 mgr.y (mgr.14152) 384 : audit 0 from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: cephadm 2023-12-07T17:15:40.964114+0000 mgr.y (mgr.14152) 385 : cephadm 1 Saving service prometheus spec with placement smithi184=a;count:1 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:40.972345+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:41.036453+0000 mon.a (mon.0) 705 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:41.044524+0000 mon.a (mon.0) 706 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:41.046336+0000 mon.a (mon.0) 707 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:41.047654+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:42.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: cluster 2023-12-07T17:15:41.050434+0000 mgr.y (mgr.14152) 386 : cluster 0 pgmap v371: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 104 B/s wr, 1 op/s 2023-12-07T17:15:42.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: audit 2023-12-07T17:15:41.055997+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[25056]: cephadm 2023-12-07T17:15:41.322201+0000 mgr.y (mgr.14152) 387 : cephadm 1 Deploying daemon prometheus.a on smithi184 2023-12-07T17:15:42.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:40.960063+0000 mgr.y (mgr.14152) 384 : audit 0 from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:42.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: cephadm 2023-12-07T17:15:40.964114+0000 mgr.y (mgr.14152) 385 : cephadm 1 Saving service prometheus spec with placement smithi184=a;count:1 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:40.972345+0000 mon.a (mon.0) 704 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:41.036453+0000 mon.a (mon.0) 705 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:41.044524+0000 mon.a (mon.0) 706 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:41.046336+0000 mon.a (mon.0) 707 : audit 0 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:41.047654+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: cluster 2023-12-07T17:15:41.050434+0000 mgr.y (mgr.14152) 386 : cluster 0 pgmap v371: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 104 B/s wr, 1 op/s 2023-12-07T17:15:42.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: audit 2023-12-07T17:15:41.055997+0000 mon.a (mon.0) 709 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:42.373 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:41 smithi064 bash[36207]: cephadm 2023-12-07T17:15:41.322201+0000 mgr.y (mgr.14152) 387 : cephadm 1 Deploying daemon prometheus.a on smithi184 2023-12-07T17:15:44.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:44 smithi064 bash[25056]: cluster 2023-12-07T17:15:43.051140+0000 mgr.y (mgr.14152) 388 : cluster 0 pgmap v372: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 780 B/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:44.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:44 smithi064 bash[36207]: cluster 2023-12-07T17:15:43.051140+0000 mgr.y (mgr.14152) 388 : cluster 0 pgmap v372: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 780 B/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:44.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:44 smithi184 bash[31336]: cluster 2023-12-07T17:15:43.051140+0000 mgr.y (mgr.14152) 388 : cluster 0 pgmap v372: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 780 B/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:46.119 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:45 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Scheduled restart job, restart counter is at 3. 2023-12-07T17:15:46.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:45 smithi064 systemd[1]: Stopped Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:46.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:45 smithi064 systemd[1]: Started Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:46.459 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:46.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:46 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:15:46.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:46 smithi184 bash[31336]: cluster 2023-12-07T17:15:45.051974+0000 mgr.y (mgr.14152) 389 : cluster 0 pgmap v373: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:46.932 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[25056]: cluster 2023-12-07T17:15:45.051974+0000 mgr.y (mgr.14152) 389 : cluster 0 pgmap v373: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:46.933 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[36207]: cluster 2023-12-07T17:15:45.051974+0000 mgr.y (mgr.14152) 389 : cluster 0 pgmap v373: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:46.933 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug 2023-12-07T17:15:46.919+0000 7f36afcff7c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:46.933 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7f36afcff7c0 time 2023-12-07T17:15:46.922659+0000 2023-12-07T17:15:46.933 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:46.933 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:46.933 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f36adb4cee7] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: main() 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: __libc_start_main() 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: _start() 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug 2023-12-07T17:15:46.923+0000 7f36afcff7c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7f36afcff7c0 time 2023-12-07T17:15:46.922659+0000 2023-12-07T17:15:46.934 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f36adb4cee7] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: main() 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: __libc_start_main() 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: _start() 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: *** Caught signal (Aborted) ** 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: in thread 7f36afcff7c0 thread_name:radosgw 2023-12-07T17:15:46.935 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f36aaeb8cf0] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: gsignal() 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: abort() 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f36adb4cf41] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:46.936 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 12: main() 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 13: __libc_start_main() 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 14: _start() 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug 2023-12-07T17:15:46.927+0000 7f36afcff7c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: in thread 7f36afcff7c0 thread_name:radosgw 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:47.370 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f36aaeb8cf0] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: gsignal() 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: abort() 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f36adb4cf41] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 12: main() 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 13: __libc_start_main() 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 14: _start() 2023-12-07T17:15:47.371 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug -6> 2023-12-07T17:15:46.919+0000 7f36afcff7c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug -1> 2023-12-07T17:15:46.923+0000 7f36afcff7c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7f36afcff7c0 time 2023-12-07T17:15:46.922659+0000 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f36adb4cee7] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.372 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: main() 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: __libc_start_main() 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: _start() 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug 0> 2023-12-07T17:15:46.927+0000 7f36afcff7c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: in thread 7f36afcff7c0 thread_name:radosgw 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f36aaeb8cf0] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: gsignal() 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: abort() 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f36adb4cf41] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.373 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 12: main() 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 13: __libc_start_main() 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 14: _start() 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug -6> 2023-12-07T17:15:46.919+0000 7f36afcff7c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug -1> 2023-12-07T17:15:46.923+0000 7f36afcff7c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7f36afcff7c0 time 2023-12-07T17:15:46.922659+0000 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f36adb4cee7] 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:47.374 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: main() 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: __libc_start_main() 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: _start() 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: debug 0> 2023-12-07T17:15:46.927+0000 7f36afcff7c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: in thread 7f36afcff7c0 thread_name:radosgw 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f36aaeb8cf0] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 2: gsignal() 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 3: abort() 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f36adb4cf41] 2023-12-07T17:15:47.375 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7f36adb4d0ad] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7f36adc3bf69] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x55ca5b29891d] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 8: (RGWRados::init_rados()+0x17a) [0x55ca5b5301fa] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x55ca5b540ba0] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x55ca5b4260ee] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 11: (rgw::AppMain::init_storage()+0x350) [0x55ca5afd8680] 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 12: main() 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 13: __libc_start_main() 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: 14: _start() 2023-12-07T17:15:47.376 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:46 smithi064 bash[91078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:48.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:47 smithi184 bash[31336]: audit 2023-12-07T17:15:46.166868+0000 mgr.y (mgr.14152) 390 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:48.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:47 smithi064 bash[25056]: audit 2023-12-07T17:15:46.166868+0000 mgr.y (mgr.14152) 390 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:48.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:47 smithi064 bash[36207]: audit 2023-12-07T17:15:46.166868+0000 mgr.y (mgr.14152) 390 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:48.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:47 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=134/n/a 2023-12-07T17:15:48.120 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:47 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:15:49.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:48 smithi184 bash[31336]: cluster 2023-12-07T17:15:47.052472+0000 mgr.y (mgr.14152) 391 : cluster 0 pgmap v374: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:49.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:48 smithi064 bash[25056]: cluster 2023-12-07T17:15:47.052472+0000 mgr.y (mgr.14152) 391 : cluster 0 pgmap v374: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:49.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:48 smithi064 bash[36207]: cluster 2023-12-07T17:15:47.052472+0000 mgr.y (mgr.14152) 391 : cluster 0 pgmap v374: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 B/s wr, 1 op/s 2023-12-07T17:15:50.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:49 smithi184 bash[31336]: cluster 2023-12-07T17:15:49.053274+0000 mgr.y (mgr.14152) 392 : cluster 0 pgmap v375: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 86 B/s wr, 2 op/s 2023-12-07T17:15:50.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:49 smithi064 bash[36207]: cluster 2023-12-07T17:15:49.053274+0000 mgr.y (mgr.14152) 392 : cluster 0 pgmap v375: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 86 B/s wr, 2 op/s 2023-12-07T17:15:50.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:49 smithi064 bash[25056]: cluster 2023-12-07T17:15:49.053274+0000 mgr.y (mgr.14152) 392 : cluster 0 pgmap v375: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 86 B/s wr, 2 op/s 2023-12-07T17:15:52.803 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled node-exporter update... 2023-12-07T17:15:52.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:52 smithi184 bash[31336]: cluster 2023-12-07T17:15:51.054015+0000 mgr.y (mgr.14152) 393 : cluster 0 pgmap v376: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:15:53.119 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:52 smithi064 bash[36207]: cluster 2023-12-07T17:15:51.054015+0000 mgr.y (mgr.14152) 393 : cluster 0 pgmap v376: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:15:53.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:52 smithi064 bash[25056]: cluster 2023-12-07T17:15:51.054015+0000 mgr.y (mgr.14152) 393 : cluster 0 pgmap v376: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:15:54.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:53 smithi184 bash[31336]: audit 2023-12-07T17:15:52.789610+0000 mgr.y (mgr.14152) 394 : audit 0 from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi064=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:54.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:53 smithi184 bash[31336]: cephadm 2023-12-07T17:15:52.793179+0000 mgr.y (mgr.14152) 395 : cephadm 1 Saving service node-exporter spec with placement smithi064=a;smithi184=b;count:2 2023-12-07T17:15:54.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:53 smithi184 bash[31336]: audit 2023-12-07T17:15:52.800062+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:54.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:53 smithi184 bash[31336]: cluster 2023-12-07T17:15:53.054727+0000 mgr.y (mgr.14152) 396 : cluster 0 pgmap v377: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[25056]: audit 2023-12-07T17:15:52.789610+0000 mgr.y (mgr.14152) 394 : audit 0 from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi064=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[25056]: cephadm 2023-12-07T17:15:52.793179+0000 mgr.y (mgr.14152) 395 : cephadm 1 Saving service node-exporter spec with placement smithi064=a;smithi184=b;count:2 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[25056]: audit 2023-12-07T17:15:52.800062+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[25056]: cluster 2023-12-07T17:15:53.054727+0000 mgr.y (mgr.14152) 396 : cluster 0 pgmap v377: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[36207]: audit 2023-12-07T17:15:52.789610+0000 mgr.y (mgr.14152) 394 : audit 0 from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi064=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[36207]: cephadm 2023-12-07T17:15:52.793179+0000 mgr.y (mgr.14152) 395 : cephadm 1 Saving service node-exporter spec with placement smithi064=a;smithi184=b;count:2 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[36207]: audit 2023-12-07T17:15:52.800062+0000 mon.a (mon.0) 710 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:54.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:53 smithi064 bash[36207]: cluster 2023-12-07T17:15:53.054727+0000 mgr.y (mgr.14152) 396 : cluster 0 pgmap v377: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:15:54.135 DEBUG:teuthology.orchestra.run.smithi064:node-exporter.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.a.service 2023-12-07T17:15:54.137 DEBUG:teuthology.orchestra.run.smithi184:node-exporter.b> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.b.service 2023-12-07T17:15:54.139 INFO:tasks.cephadm:Adding alertmanager.a on smithi064 2023-12-07T17:15:54.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply alertmanager '1;smithi064=a' 2023-12-07T17:15:54.197 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:15:54.236 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:15:56.067 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:56 smithi184 systemd[1]: Started Ceph prometheus.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:56.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:15:57.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[31336]: cluster 2023-12-07T17:15:55.055541+0000 mgr.y (mgr.14152) 397 : cluster 0 pgmap v378: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2023-12-07T17:15:57.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[31336]: audit 2023-12-07T17:15:56.086509+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[31336]: audit 2023-12-07T17:15:56.093483+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[31336]: audit 2023-12-07T17:15:56.140110+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.000 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:56 smithi184 bash[31336]: audit 2023-12-07T17:15:56.146138+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[25056]: cluster 2023-12-07T17:15:55.055541+0000 mgr.y (mgr.14152) 397 : cluster 0 pgmap v378: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[25056]: audit 2023-12-07T17:15:56.086509+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[25056]: audit 2023-12-07T17:15:56.093483+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[25056]: audit 2023-12-07T17:15:56.140110+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[25056]: audit 2023-12-07T17:15:56.146138+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[36207]: cluster 2023-12-07T17:15:55.055541+0000 mgr.y (mgr.14152) 397 : cluster 0 pgmap v378: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[36207]: audit 2023-12-07T17:15:56.086509+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[36207]: audit 2023-12-07T17:15:56.093483+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[36207]: audit 2023-12-07T17:15:56.140110+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' 2023-12-07T17:15:57.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:56 smithi064 bash[36207]: audit 2023-12-07T17:15:56.146138+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-12-07T17:15:57.317 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[32290]: ignoring --setuser ceph since I am not root 2023-12-07T17:15:57.317 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[32290]: ignoring --setgroup ceph since I am not root 2023-12-07T17:15:57.317 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[32290]: debug 2023-12-07T17:15:57.205+0000 7f062df61700 1 -- 172.21.15.184:0/746476838 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9eb6fe1a0 con 0x55c9ec47a800 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.166Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.166Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.166Z caller=main.go:570 level=info host_details="(Linux 5.15.0-89-generic #99~20.04.1-Ubuntu SMP Thu Nov 2 15:16:47 UTC 2023 x86_64 smithi184 (none))" 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.166Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.166Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.167Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.168Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.169Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2023-12-07T17:15:57.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.169Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.171Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.171Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.931µs 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.171Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.172Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.172Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=39.841µs wal_replay_duration=1.182998ms wbl_replay_duration=146ns total_replay_duration=1.246092ms 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.174Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.175Z caller=main.go:1029 level=info msg="TSDB started" 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.175Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.183Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=mgr-prometheus\": read tcp 172.21.15.184:48462->172.21.15.64:8765: read: connection reset by peer" 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.183Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=ceph-exporter\": read tcp 172.21.15.184:48458->172.21.15.64:8765: read: connection reset by peer" 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.203Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=28.031205ms db_storage=1.156µs remote_storage=2.024µs web_handler=526ns query_engine=870ns scrape=448.568µs scrape_sd=139.379µs notify=1.53µs notify_sd=8.654µs rules=27.051191ms tracing=10.835µs 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.203Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2023-12-07T17:15:57.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[80576]: ts=2023-12-07T17:15:57.203Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2023-12-07T17:15:57.620 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25333]: ignoring --setuser ceph since I am not root 2023-12-07T17:15:57.620 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25333]: ignoring --setgroup ceph since I am not root 2023-12-07T17:15:57.620 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25333]: debug 2023-12-07T17:15:57.375+0000 7fa23b9ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:15:57.716 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[32290]: debug 2023-12-07T17:15:57.361+0000 7f0637cf3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T17:15:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[31336]: audit 2023-12-07T17:15:56.175239+0000 mgr.y (mgr.14152) 398 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[31336]: audit 2023-12-07T17:15:57.140862+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-12-07T17:15:58.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:15:57 smithi184 bash[31336]: cluster 2023-12-07T17:15:57.151755+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2023-12-07T17:15:58.098 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[36207]: audit 2023-12-07T17:15:56.175239+0000 mgr.y (mgr.14152) 398 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:58.098 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[36207]: audit 2023-12-07T17:15:57.140862+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-12-07T17:15:58.098 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[36207]: cluster 2023-12-07T17:15:57.151755+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2023-12-07T17:15:58.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25056]: audit 2023-12-07T17:15:56.175239+0000 mgr.y (mgr.14152) 398 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:15:58.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25056]: audit 2023-12-07T17:15:57.140862+0000 mon.a (mon.0) 715 : audit 1 from='mgr.14152 172.21.15.64:0/2116901826' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-12-07T17:15:58.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:15:57 smithi064 bash[25056]: cluster 2023-12-07T17:15:57.151755+0000 mon.a (mon.0) 716 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2023-12-07T17:15:58.099 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:57 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Scheduled restart job, restart counter is at 4. 2023-12-07T17:15:58.099 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:57 smithi064 systemd[1]: Stopped Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:58.099 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:57 smithi064 systemd[1]: Started Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:15:58.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:58 smithi064 bash[25333]: debug 2023-12-07T17:15:58.343+0000 7fa23b9ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:15:58.817 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:58 smithi184 bash[32290]: debug 2023-12-07T17:15:58.337+0000 7f0637cf3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T17:15:58.817 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:58 smithi184 bash[32290]: debug 2023-12-07T17:15:58.445+0000 7f0637cf3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:15:58.817 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:58 smithi184 bash[32290]: debug 2023-12-07T17:15:58.553+0000 7f0637cf3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:15:58.843 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:58 smithi064 bash[25333]: debug 2023-12-07T17:15:58.447+0000 7fa23b9ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T17:15:58.843 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:58 smithi064 bash[25333]: debug 2023-12-07T17:15:58.559+0000 7fa23b9ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T17:15:58.926 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:15:59.119 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:58 smithi064 bash[25333]: debug 2023-12-07T17:15:58.839+0000 7fa23b9ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:15:59.205 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:58 smithi184 bash[32290]: debug 2023-12-07T17:15:58.829+0000 7f0637cf3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:15:59.518 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[25333]: debug 2023-12-07T17:15:59.231+0000 7fa23b9ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:15:59.518 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[25333]: debug 2023-12-07T17:15:59.411+0000 7fa23b9ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:15:59.518 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fe6e4af77c0 time 2023-12-07T17:15:59.290374+0000 2023-12-07T17:15:59.518 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:59.518 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug 2023-12-07T17:15:59.287+0000 7fe6e4af77c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:59.518 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.518 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe6e2944ee7] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: main() 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: __libc_start_main() 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: _start() 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug 2023-12-07T17:15:59.291+0000 7fe6e4af77c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fe6e4af77c0 time 2023-12-07T17:15:59.290374+0000 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.519 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe6e2944ee7] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.520 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: main() 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: __libc_start_main() 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: _start() 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: *** Caught signal (Aborted) ** 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: in thread 7fe6e4af77c0 thread_name:radosgw 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fe6dfcb0cf0] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: gsignal() 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: abort() 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe6e2944f41] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.521 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 12: main() 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 13: __libc_start_main() 2023-12-07T17:15:59.522 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 14: _start() 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug 2023-12-07T17:15:59.295+0000 7fe6e4af77c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: in thread 7fe6e4af77c0 thread_name:radosgw 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fe6dfcb0cf0] 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: gsignal() 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: abort() 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe6e2944f41] 2023-12-07T17:15:59.525 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 12: main() 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 13: __libc_start_main() 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 14: _start() 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug -5> 2023-12-07T17:15:59.287+0000 7fe6e4af77c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug -1> 2023-12-07T17:15:59.291+0000 7fe6e4af77c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fe6e4af77c0 time 2023-12-07T17:15:59.290374+0000 2023-12-07T17:15:59.526 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe6e2944ee7] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: main() 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: __libc_start_main() 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: _start() 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug 0> 2023-12-07T17:15:59.295+0000 7fe6e4af77c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: in thread 7fe6e4af77c0 thread_name:radosgw 2023-12-07T17:15:59.527 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fe6dfcb0cf0] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: gsignal() 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: abort() 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe6e2944f41] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 12: main() 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 13: __libc_start_main() 2023-12-07T17:15:59.528 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 14: _start() 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug -5> 2023-12-07T17:15:59.287+0000 7fe6e4af77c0 -1 ERROR: fail to register admin socket command (r=-17) 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug -1> 2023-12-07T17:15:59.291+0000 7fe6e4af77c0 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: In function 'virtual ceph::common::RefCountedObject::~RefCountedObject()' thread 7fe6e4af77c0 time 2023-12-07T17:15:59.290374+0000 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-64-ge068ebcf/rpm/el8/BUILD/ceph-19.0.0-64-ge068ebcf/src/common/RefCountedObj.cc: 14: FAILED ceph_assert(nref == 0) 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe6e2944ee7] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: main() 2023-12-07T17:15:59.529 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: __libc_start_main() 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: _start() 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: debug 0> 2023-12-07T17:15:59.295+0000 7fe6e4af77c0 -1 *** Caught signal (Aborted) ** 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: in thread 7fe6e4af77c0 thread_name:radosgw 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: ceph version 19.0.0-64-ge068ebcf (e068ebcf7ecc6503f24666fb6b152034d3fe1067) reef (dev) 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fe6dfcb0cf0] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 2: gsignal() 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 3: abort() 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe6e2944f41] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba0ad) [0x7fe6e29450ad] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 6: /usr/lib64/ceph/libceph-common.so.2(+0x3a8f69) [0x7fe6e2a33f69] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 7: (RGWCoroutinesManagerRegistry::~RGWCoroutinesManagerRegistry()+0xd) [0x56060577b91d] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 8: (RGWRados::init_rados()+0x17a) [0x560605a131fa] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 9: (RGWRados::init_begin(DoutPrefixProvider const*)+0x180) [0x560605a23ba0] 2023-12-07T17:15:59.530 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 10: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, boost::asio::io_context&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0xc8e) [0x5606059090ee] 2023-12-07T17:15:59.531 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 11: (rgw::AppMain::init_storage()+0x350) [0x5606054bb680] 2023-12-07T17:15:59.531 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 12: main() 2023-12-07T17:15:59.531 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 13: __libc_start_main() 2023-12-07T17:15:59.531 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: 14: _start() 2023-12-07T17:15:59.531 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[91805]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-07T17:15:59.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:59 smithi184 bash[32290]: debug 2023-12-07T17:15:59.201+0000 7f0637cf3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T17:15:59.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:59 smithi184 bash[32290]: debug 2023-12-07T17:15:59.361+0000 7f0637cf3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T17:15:59.847 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:15:59 smithi184 bash[32290]: debug 2023-12-07T17:15:59.577+0000 7f0637cf3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:15:59.869 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:15:59 smithi064 bash[25333]: debug 2023-12-07T17:15:59.639+0000 7fa23b9ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T17:16:00.619 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:00 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=134/n/a 2023-12-07T17:16:00.620 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:00 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:16:01.067 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:00 smithi184 bash[32290]: debug 2023-12-07T17:16:00.593+0000 7f0637cf3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:16:01.067 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:00 smithi184 bash[32290]: debug 2023-12-07T17:16:00.709+0000 7f0637cf3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:16:01.067 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:00 smithi184 bash[32290]: debug 2023-12-07T17:16:00.837+0000 7f0637cf3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:16:01.119 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:00 smithi064 bash[25333]: debug 2023-12-07T17:16:00.763+0000 7fa23b9ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T17:16:01.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:00 smithi064 bash[25333]: debug 2023-12-07T17:16:00.879+0000 7fa23b9ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T17:16:01.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:01 smithi064 bash[25333]: debug 2023-12-07T17:16:01.003+0000 7fa23b9ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T17:16:01.647 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:01 smithi184 bash[32290]: debug 2023-12-07T17:16:01.357+0000 7f0637cf3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:16:01.647 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:01 smithi184 bash[32290]: debug 2023-12-07T17:16:01.601+0000 7f0637cf3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:16:01.869 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:01 smithi064 bash[25333]: debug 2023-12-07T17:16:01.491+0000 7fa23b9ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T17:16:01.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:01 smithi064 bash[25333]: debug 2023-12-07T17:16:01.727+0000 7fa23b9ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T17:16:02.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:02 smithi184 bash[32290]: debug 2023-12-07T17:16:02.097+0000 7f0637cf3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:16:02.619 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:02 smithi064 bash[25333]: debug 2023-12-07T17:16:02.199+0000 7fa23b9ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T17:16:03.067 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:02 smithi184 bash[32290]: debug 2023-12-07T17:16:02.617+0000 7f0637cf3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:16:03.119 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:02 smithi064 bash[25333]: debug 2023-12-07T17:16:02.711+0000 7fa23b9ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T17:16:03.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.161+0000 7f0637cf3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:16:03.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.273+0000 7f0637cf3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:16:03.620 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:03 smithi064 bash[25333]: debug 2023-12-07T17:16:03.299+0000 7fa23b9ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T17:16:03.620 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:03 smithi064 bash[25333]: debug 2023-12-07T17:16:03.415+0000 7fa23b9ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T17:16:03.875 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.617+0000 7f0637cf3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:16:03.875 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.729+0000 7f0637cf3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:16:04.043 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:03 smithi064 bash[25333]: debug 2023-12-07T17:16:03.771+0000 7fa23b9ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T17:16:04.043 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:03 smithi064 bash[25333]: debug 2023-12-07T17:16:03.891+0000 7fa23b9ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T17:16:04.295 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:04 smithi064 bash[25333]: debug 2023-12-07T17:16:04.039+0000 7fa23b9ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:16:04.295 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:04 smithi064 bash[25333]: debug 2023-12-07T17:16:04.159+0000 7fa23b9ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:16:04.302 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.869+0000 7f0637cf3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T17:16:04.302 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:03 smithi184 bash[32290]: debug 2023-12-07T17:16:03.989+0000 7f0637cf3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T17:16:04.302 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:04 smithi184 bash[32290]: debug 2023-12-07T17:16:04.109+0000 7f0637cf3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:16:04.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:04 smithi184 bash[32290]: debug 2023-12-07T17:16:04.297+0000 7f0637cf3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:16:04.567 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:04 smithi184 bash[32290]: debug 2023-12-07T17:16:04.421+0000 7f0637cf3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:16:04.619 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:04 smithi064 bash[25333]: debug 2023-12-07T17:16:04.291+0000 7fa23b9ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T17:16:04.619 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:04 smithi064 bash[25333]: debug 2023-12-07T17:16:04.491+0000 7fa23b9ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T17:16:05.119 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:04 smithi064 bash[25333]: debug 2023-12-07T17:16:04.627+0000 7fa23b9ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T17:16:06.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.665723+0000 mon.a (mon.0) 717 : cluster 0 Standby manager daemon x restarted 2023-12-07T17:16:06.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.665885+0000 mon.a (mon.0) 718 : cluster 0 Standby manager daemon x started 2023-12-07T17:16:06.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: audit 2023-12-07T17:16:05.669939+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: audit 2023-12-07T17:16:05.672132+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: audit 2023-12-07T17:16:05.677020+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[31336]: audit 2023-12-07T17:16:05.677565+0000 mon.b (mon.1) 32 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: debug 2023-12-07T17:16:05.641+0000 7f0637cf3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: [07/Dec/2023:17:16:05] ENGINE Bus STARTING 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: CherryPy Checker: 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: The Application mounted at '' has an empty config. 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: [07/Dec/2023:17:16:05] ENGINE Serving on http://:::9283 2023-12-07T17:16:06.068 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:16:05 smithi184 bash[32290]: [07/Dec/2023:17:16:05] ENGINE Bus STARTED 2023-12-07T17:16:06.096 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.665723+0000 mon.a (mon.0) 717 : cluster 0 Standby manager daemon x restarted 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.665885+0000 mon.a (mon.0) 718 : cluster 0 Standby manager daemon x started 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: audit 2023-12-07T17:16:05.669939+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: audit 2023-12-07T17:16:05.672132+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: audit 2023-12-07T17:16:05.677020+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[36207]: audit 2023-12-07T17:16:05.677565+0000 mon.b (mon.1) 32 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.665723+0000 mon.a (mon.0) 717 : cluster 0 Standby manager daemon x restarted 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.665885+0000 mon.a (mon.0) 718 : cluster 0 Standby manager daemon x started 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: audit 2023-12-07T17:16:05.669939+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: audit 2023-12-07T17:16:05.672132+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T17:16:06.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: audit 2023-12-07T17:16:05.677020+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2023-12-07T17:16:06.098 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25056]: audit 2023-12-07T17:16:05.677565+0000 mon.b (mon.1) 32 : audit 0 from='mgr.? 172.21.15.184:0/235353903' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T17:16:06.098 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:05 smithi064 bash[25333]: debug 2023-12-07T17:16:05.927+0000 7fa23b9ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T17:16:06.098 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25333]: [07/Dec/2023:17:16:06] ENGINE Bus STARTING 2023-12-07T17:16:06.369 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25333]: CherryPy Checker: 2023-12-07T17:16:06.369 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25333]: The Application mounted at '' has an empty config. 2023-12-07T17:16:06.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25333]: [07/Dec/2023:17:16:06] ENGINE Serving on http://:::9283 2023-12-07T17:16:06.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25333]: [07/Dec/2023:17:16:06] ENGINE Bus STARTED 2023-12-07T17:16:06.990 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled alertmanager update... 2023-12-07T17:16:07.047 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:07.047 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.727051+0000 mon.a (mon.0) 719 : cluster 0 mgrmap e18: y(active, since 10m), standbys: x 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.951840+0000 mon.a (mon.0) 720 : cluster 1 Active manager daemon y restarted 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.952424+0000 mon.a (mon.0) 721 : cluster 1 Activating manager daemon y 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.964039+0000 mon.a (mon.0) 722 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:05.964671+0000 mon.a (mon.0) 723 : cluster 0 mgrmap e19: y(active, starting, since 0.0124388s), standbys: x 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.968404+0000 mon.c (mon.2) 12 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.968635+0000 mon.c (mon.2) 13 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.968953+0000 mon.c (mon.2) 14 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.970637+0000 mon.c (mon.2) 15 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.970889+0000 mon.c (mon.2) 16 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.971203+0000 mon.c (mon.2) 17 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.971569+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.971935+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.972341+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:16:07.048 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.972737+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.973043+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.973322+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.973596+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.974021+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.974290+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:05.974896+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: cluster 2023-12-07T17:16:06.007333+0000 mon.a (mon.0) 724 : cluster 1 Manager daemon y is now available 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.053187+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.059395+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.080327+0000 mon.a (mon.0) 725 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.098859+0000 mon.a (mon.0) 726 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.101734+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.106522+0000 mon.c (mon.2) 31 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.106834+0000 mon.a (mon.0) 727 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.142323+0000 mon.c (mon.2) 32 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.050 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:06 smithi184 bash[31336]: audit 2023-12-07T17:16:06.142827+0000 mon.a (mon.0) 728 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.727051+0000 mon.a (mon.0) 719 : cluster 0 mgrmap e18: y(active, since 10m), standbys: x 2023-12-07T17:16:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.951840+0000 mon.a (mon.0) 720 : cluster 1 Active manager daemon y restarted 2023-12-07T17:16:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.952424+0000 mon.a (mon.0) 721 : cluster 1 Activating manager daemon y 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.964039+0000 mon.a (mon.0) 722 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:05.964671+0000 mon.a (mon.0) 723 : cluster 0 mgrmap e19: y(active, starting, since 0.0124388s), standbys: x 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.968404+0000 mon.c (mon.2) 12 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.968635+0000 mon.c (mon.2) 13 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.968953+0000 mon.c (mon.2) 14 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.970637+0000 mon.c (mon.2) 15 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.970889+0000 mon.c (mon.2) 16 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.971203+0000 mon.c (mon.2) 17 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.971569+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.971935+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.972341+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.972737+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:16:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.973043+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.973322+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.973596+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.974021+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.974290+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.727051+0000 mon.a (mon.0) 719 : cluster 0 mgrmap e18: y(active, since 10m), standbys: x 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.951840+0000 mon.a (mon.0) 720 : cluster 1 Active manager daemon y restarted 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.952424+0000 mon.a (mon.0) 721 : cluster 1 Activating manager daemon y 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.964039+0000 mon.a (mon.0) 722 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2023-12-07T17:16:07.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:05.964671+0000 mon.a (mon.0) 723 : cluster 0 mgrmap e19: y(active, starting, since 0.0124388s), standbys: x 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.968404+0000 mon.c (mon.2) 12 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.968635+0000 mon.c (mon.2) 13 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.968953+0000 mon.c (mon.2) 14 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.970637+0000 mon.c (mon.2) 15 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.970889+0000 mon.c (mon.2) 16 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.971203+0000 mon.c (mon.2) 17 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.971569+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.971935+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.972341+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.972737+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.973043+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.973322+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T17:16:07.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.973596+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.974021+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.974290+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:05.974896+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: cluster 2023-12-07T17:16:06.007333+0000 mon.a (mon.0) 724 : cluster 1 Manager daemon y is now available 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.053187+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.059395+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.080327+0000 mon.a (mon.0) 725 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.098859+0000 mon.a (mon.0) 726 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.101734+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.106522+0000 mon.c (mon.2) 31 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.106834+0000 mon.a (mon.0) 727 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.142323+0000 mon.c (mon.2) 32 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[25056]: audit 2023-12-07T17:16:06.142827+0000 mon.a (mon.0) 728 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:05.974896+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: cluster 2023-12-07T17:16:06.007333+0000 mon.a (mon.0) 724 : cluster 1 Manager daemon y is now available 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.053187+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.059395+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.080327+0000 mon.a (mon.0) 725 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.098859+0000 mon.a (mon.0) 726 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.101734+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:07.125 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.106522+0000 mon.c (mon.2) 31 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.126 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.106834+0000 mon.a (mon.0) 727 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2023-12-07T17:16:07.126 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.142323+0000 mon.c (mon.2) 32 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.126 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:06 smithi064 bash[36207]: audit 2023-12-07T17:16:06.142827+0000 mon.a (mon.0) 728 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2023-12-07T17:16:07.924 DEBUG:teuthology.orchestra.run.smithi064:alertmanager.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@alertmanager.a.service 2023-12-07T17:16:07.927 INFO:tasks.cephadm:Adding grafana.a on smithi184 2023-12-07T17:16:07.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph orch apply grafana '1;smithi184=a' 2023-12-07T17:16:08.164 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:16:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cephadm 2023-12-07T17:16:06.576443+0000 mgr.y (mgr.24626) 1 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTING 2023-12-07T17:16:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cephadm 2023-12-07T17:16:06.679038+0000 mgr.y (mgr.24626) 2 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on http://172.21.15.64:8765 2023-12-07T17:16:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cephadm 2023-12-07T17:16:06.809570+0000 mgr.y (mgr.24626) 3 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on https://172.21.15.64:7150 2023-12-07T17:16:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cephadm 2023-12-07T17:16:06.809977+0000 mgr.y (mgr.24626) 4 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTED 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cluster 2023-12-07T17:16:06.972721+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e20: y(active, since 1.02049s), standbys: x 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: audit 2023-12-07T17:16:06.980053+0000 mgr.y (mgr.24626) 5 : audit 0 from='client.24637 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: audit 2023-12-07T17:16:06.980136+0000 mgr.y (mgr.24626) 6 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cluster 2023-12-07T17:16:06.982878+0000 mgr.y (mgr.24626) 7 : cluster 0 pgmap v3: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: cephadm 2023-12-07T17:16:06.983141+0000 mgr.y (mgr.24626) 8 : cephadm 1 Saving service alertmanager spec with placement smithi064=a;count:1 2023-12-07T17:16:08.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:07 smithi184 bash[31336]: audit 2023-12-07T17:16:06.987363+0000 mon.a (mon.0) 730 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cephadm 2023-12-07T17:16:06.576443+0000 mgr.y (mgr.24626) 1 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTING 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cephadm 2023-12-07T17:16:06.679038+0000 mgr.y (mgr.24626) 2 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on http://172.21.15.64:8765 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cephadm 2023-12-07T17:16:06.809570+0000 mgr.y (mgr.24626) 3 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on https://172.21.15.64:7150 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cephadm 2023-12-07T17:16:06.809977+0000 mgr.y (mgr.24626) 4 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTED 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cluster 2023-12-07T17:16:06.972721+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e20: y(active, since 1.02049s), standbys: x 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: audit 2023-12-07T17:16:06.980053+0000 mgr.y (mgr.24626) 5 : audit 0 from='client.24637 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: audit 2023-12-07T17:16:06.980136+0000 mgr.y (mgr.24626) 6 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cluster 2023-12-07T17:16:06.982878+0000 mgr.y (mgr.24626) 7 : cluster 0 pgmap v3: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: cephadm 2023-12-07T17:16:06.983141+0000 mgr.y (mgr.24626) 8 : cephadm 1 Saving service alertmanager spec with placement smithi064=a;count:1 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[25056]: audit 2023-12-07T17:16:06.987363+0000 mon.a (mon.0) 730 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cephadm 2023-12-07T17:16:06.576443+0000 mgr.y (mgr.24626) 1 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTING 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cephadm 2023-12-07T17:16:06.679038+0000 mgr.y (mgr.24626) 2 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on http://172.21.15.64:8765 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cephadm 2023-12-07T17:16:06.809570+0000 mgr.y (mgr.24626) 3 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Serving on https://172.21.15.64:7150 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cephadm 2023-12-07T17:16:06.809977+0000 mgr.y (mgr.24626) 4 : cephadm 1 [07/Dec/2023:17:16:06] ENGINE Bus STARTED 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cluster 2023-12-07T17:16:06.972721+0000 mon.a (mon.0) 729 : cluster 0 mgrmap e20: y(active, since 1.02049s), standbys: x 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: audit 2023-12-07T17:16:06.980053+0000 mgr.y (mgr.24626) 5 : audit 0 from='client.24637 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi064=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: audit 2023-12-07T17:16:06.980136+0000 mgr.y (mgr.24626) 6 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cluster 2023-12-07T17:16:06.982878+0000 mgr.y (mgr.24626) 7 : cluster 0 pgmap v3: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: cephadm 2023-12-07T17:16:06.983141+0000 mgr.y (mgr.24626) 8 : cephadm 1 Saving service alertmanager spec with placement smithi064=a;count:1 2023-12-07T17:16:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:07 smithi064 bash[36207]: audit 2023-12-07T17:16:06.987363+0000 mon.a (mon.0) 730 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:09.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:08 smithi184 bash[31336]: cluster 2023-12-07T17:16:07.964856+0000 mon.a (mon.0) 731 : cluster 1 Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-12-07T17:16:09.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:08 smithi184 bash[31336]: cluster 2023-12-07T17:16:07.964929+0000 mon.a (mon.0) 732 : cluster 1 Cluster is now healthy 2023-12-07T17:16:09.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:08 smithi184 bash[31336]: cluster 2023-12-07T17:16:07.972504+0000 mgr.y (mgr.24626) 9 : cluster 0 pgmap v4: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:09.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[25056]: cluster 2023-12-07T17:16:07.964856+0000 mon.a (mon.0) 731 : cluster 1 Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-12-07T17:16:09.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[25056]: cluster 2023-12-07T17:16:07.964929+0000 mon.a (mon.0) 732 : cluster 1 Cluster is now healthy 2023-12-07T17:16:09.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[25056]: cluster 2023-12-07T17:16:07.972504+0000 mgr.y (mgr.24626) 9 : cluster 0 pgmap v4: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:09.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[36207]: cluster 2023-12-07T17:16:07.964856+0000 mon.a (mon.0) 731 : cluster 1 Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2023-12-07T17:16:09.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[36207]: cluster 2023-12-07T17:16:07.964929+0000 mon.a (mon.0) 732 : cluster 1 Cluster is now healthy 2023-12-07T17:16:09.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:08 smithi064 bash[36207]: cluster 2023-12-07T17:16:07.972504+0000 mgr.y (mgr.24626) 9 : cluster 0 pgmap v4: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:10.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:09 smithi184 bash[31336]: cluster 2023-12-07T17:16:08.979541+0000 mon.a (mon.0) 733 : cluster 0 mgrmap e21: y(active, since 3s), standbys: x 2023-12-07T17:16:10.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:09 smithi064 bash[25056]: cluster 2023-12-07T17:16:08.979541+0000 mon.a (mon.0) 733 : cluster 0 mgrmap e21: y(active, since 3s), standbys: x 2023-12-07T17:16:10.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:09 smithi064 bash[36207]: cluster 2023-12-07T17:16:08.979541+0000 mon.a (mon.0) 733 : cluster 0 mgrmap e21: y(active, since 3s), standbys: x 2023-12-07T17:16:10.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:10 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Scheduled restart job, restart counter is at 5. 2023-12-07T17:16:10.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:10 smithi064 systemd[1]: Stopped Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:16:10.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:10 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Start request repeated too quickly. 2023-12-07T17:16:10.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:10 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2023-12-07T17:16:10.870 INFO:journalctl@ceph.rgw.foo.a.smithi064.stdout:Dec 07 17:16:10 smithi064 systemd[1]: Failed to start Ceph rgw.foo.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:16:11.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:11 smithi184 bash[31336]: cluster 2023-12-07T17:16:09.972985+0000 mgr.y (mgr.24626) 10 : cluster 0 pgmap v5: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:11.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:11 smithi064 bash[25056]: cluster 2023-12-07T17:16:09.972985+0000 mgr.y (mgr.24626) 10 : cluster 0 pgmap v5: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:11.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:11 smithi064 bash[36207]: cluster 2023-12-07T17:16:09.972985+0000 mgr.y (mgr.24626) 10 : cluster 0 pgmap v5: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:12.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:12 smithi184 bash[31336]: cluster 2023-12-07T17:16:11.001831+0000 mon.a (mon.0) 734 : cluster 0 mgrmap e22: y(active, since 5s), standbys: x 2023-12-07T17:16:12.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:12 smithi064 bash[36207]: cluster 2023-12-07T17:16:11.001831+0000 mon.a (mon.0) 734 : cluster 0 mgrmap e22: y(active, since 5s), standbys: x 2023-12-07T17:16:12.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:12 smithi064 bash[25056]: cluster 2023-12-07T17:16:11.001831+0000 mon.a (mon.0) 734 : cluster 0 mgrmap e22: y(active, since 5s), standbys: x 2023-12-07T17:16:12.492 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:16:13.515 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:13 smithi184 bash[31336]: cluster 2023-12-07T17:16:11.973544+0000 mgr.y (mgr.24626) 11 : cluster 0 pgmap v6: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:13.515 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:13 smithi184 bash[31336]: audit 2023-12-07T17:16:12.157533+0000 mon.a (mon.0) 735 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.515 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:13 smithi184 bash[31336]: audit 2023-12-07T17:16:12.167539+0000 mon.a (mon.0) 736 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.515 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:13 smithi184 bash[31336]: audit 2023-12-07T17:16:12.551351+0000 mon.a (mon.0) 737 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.515 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:13 smithi184 bash[31336]: audit 2023-12-07T17:16:12.559887+0000 mon.a (mon.0) 738 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[25056]: cluster 2023-12-07T17:16:11.973544+0000 mgr.y (mgr.24626) 11 : cluster 0 pgmap v6: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[25056]: audit 2023-12-07T17:16:12.157533+0000 mon.a (mon.0) 735 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[25056]: audit 2023-12-07T17:16:12.167539+0000 mon.a (mon.0) 736 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[25056]: audit 2023-12-07T17:16:12.551351+0000 mon.a (mon.0) 737 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[25056]: audit 2023-12-07T17:16:12.559887+0000 mon.a (mon.0) 738 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[36207]: cluster 2023-12-07T17:16:11.973544+0000 mgr.y (mgr.24626) 11 : cluster 0 pgmap v6: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[36207]: audit 2023-12-07T17:16:12.157533+0000 mon.a (mon.0) 735 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[36207]: audit 2023-12-07T17:16:12.167539+0000 mon.a (mon.0) 736 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[36207]: audit 2023-12-07T17:16:12.551351+0000 mon.a (mon.0) 737 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:13.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:13 smithi064 bash[36207]: audit 2023-12-07T17:16:12.559887+0000 mon.a (mon.0) 738 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.222577+0000 mon.a (mon.0) 739 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.235930+0000 mon.a (mon.0) 740 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.241485+0000 mon.c (mon.2) 33 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.242072+0000 mon.a (mon.0) 741 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.243707+0000 mon.c (mon.2) 34 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.244228+0000 mon.a (mon.0) 742 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.245789+0000 mon.c (mon.2) 35 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.246276+0000 mon.a (mon.0) 743 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.247718+0000 mon.c (mon.2) 36 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.248144+0000 mon.a (mon.0) 744 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.256808+0000 mon.a (mon.0) 745 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.614685+0000 mon.a (mon.0) 746 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.630159+0000 mon.a (mon.0) 747 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.635502+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.636127+0000 mon.a (mon.0) 748 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.637977+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.570 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.638604+0000 mon.a (mon.0) 749 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.640274+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.640934+0000 mon.a (mon.0) 750 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.642574+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.643227+0000 mon.a (mon.0) 751 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.650339+0000 mon.a (mon.0) 752 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.653465+0000 mon.c (mon.2) 41 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:14.571 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:14 smithi184 bash[31336]: audit 2023-12-07T17:16:13.654614+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.222577+0000 mon.a (mon.0) 739 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.235930+0000 mon.a (mon.0) 740 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.241485+0000 mon.c (mon.2) 33 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.242072+0000 mon.a (mon.0) 741 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.243707+0000 mon.c (mon.2) 34 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.244228+0000 mon.a (mon.0) 742 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.245789+0000 mon.c (mon.2) 35 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.246276+0000 mon.a (mon.0) 743 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.247718+0000 mon.c (mon.2) 36 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.248144+0000 mon.a (mon.0) 744 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.256808+0000 mon.a (mon.0) 745 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.614685+0000 mon.a (mon.0) 746 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.630159+0000 mon.a (mon.0) 747 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.635502+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.636127+0000 mon.a (mon.0) 748 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.637977+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.638604+0000 mon.a (mon.0) 749 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.640274+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.640934+0000 mon.a (mon.0) 750 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.642574+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.643227+0000 mon.a (mon.0) 751 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.650339+0000 mon.a (mon.0) 752 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.653465+0000 mon.c (mon.2) 41 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[36207]: audit 2023-12-07T17:16:13.654614+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.222577+0000 mon.a (mon.0) 739 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.235930+0000 mon.a (mon.0) 740 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.241485+0000 mon.c (mon.2) 33 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.242072+0000 mon.a (mon.0) 741 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.243707+0000 mon.c (mon.2) 34 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.244228+0000 mon.a (mon.0) 742 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.245789+0000 mon.c (mon.2) 35 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.246276+0000 mon.a (mon.0) 743 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.247718+0000 mon.c (mon.2) 36 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.248144+0000 mon.a (mon.0) 744 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.256808+0000 mon.a (mon.0) 745 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.614685+0000 mon.a (mon.0) 746 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.630159+0000 mon.a (mon.0) 747 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.635502+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.636127+0000 mon.a (mon.0) 748 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.637977+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.638604+0000 mon.a (mon.0) 749 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.640274+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.640934+0000 mon.a (mon.0) 750 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.642574+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.643227+0000 mon.a (mon.0) 751 : audit 1 from='mgr.24626 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T17:16:14.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.650339+0000 mon.a (mon.0) 752 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:14.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.653465+0000 mon.c (mon.2) 41 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:14.624 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:14 smithi064 bash[25056]: audit 2023-12-07T17:16:13.654614+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:15.379 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.249367+0000 mgr.y (mgr.24626) 12 : cephadm 1 Adjusting osd_memory_target on smithi064 to 3804M 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.644489+0000 mgr.y (mgr.24626) 13 : cephadm 1 Adjusting osd_memory_target on smithi184 to 3804M 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.656569+0000 mgr.y (mgr.24626) 14 : cephadm 1 Updating smithi064:/etc/ceph/ceph.conf 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.656959+0000 mgr.y (mgr.24626) 15 : cephadm 1 Updating smithi184:/etc/ceph/ceph.conf 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.821525+0000 mgr.y (mgr.24626) 16 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.972998+0000 mgr.y (mgr.24626) 17 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cluster 2023-12-07T17:16:13.974311+0000 mgr.y (mgr.24626) 18 : cluster 0 pgmap v7: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:13.986296+0000 mgr.y (mgr.24626) 19 : cephadm 1 Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:14.137742+0000 mgr.y (mgr.24626) 20 : cephadm 1 Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cephadm 2023-12-07T17:16:14.151976+0000 mgr.y (mgr.24626) 21 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: audit 2023-12-07T17:16:14.331819+0000 mon.a (mon.0) 753 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: audit 2023-12-07T17:16:14.344117+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: audit 2023-12-07T17:16:14.531905+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: audit 2023-12-07T17:16:14.544092+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: audit 2023-12-07T17:16:14.555405+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[25056]: cluster 2023-12-07T17:16:14.646834+0000 mon.a (mon.0) 758 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.249367+0000 mgr.y (mgr.24626) 12 : cephadm 1 Adjusting osd_memory_target on smithi064 to 3804M 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.644489+0000 mgr.y (mgr.24626) 13 : cephadm 1 Adjusting osd_memory_target on smithi184 to 3804M 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.656569+0000 mgr.y (mgr.24626) 14 : cephadm 1 Updating smithi064:/etc/ceph/ceph.conf 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.656959+0000 mgr.y (mgr.24626) 15 : cephadm 1 Updating smithi184:/etc/ceph/ceph.conf 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.821525+0000 mgr.y (mgr.24626) 16 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.972998+0000 mgr.y (mgr.24626) 17 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cluster 2023-12-07T17:16:13.974311+0000 mgr.y (mgr.24626) 18 : cluster 0 pgmap v7: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:13.986296+0000 mgr.y (mgr.24626) 19 : cephadm 1 Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.381 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:14.137742+0000 mgr.y (mgr.24626) 20 : cephadm 1 Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cephadm 2023-12-07T17:16:14.151976+0000 mgr.y (mgr.24626) 21 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: audit 2023-12-07T17:16:14.331819+0000 mon.a (mon.0) 753 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: audit 2023-12-07T17:16:14.344117+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: audit 2023-12-07T17:16:14.531905+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: audit 2023-12-07T17:16:14.544092+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: audit 2023-12-07T17:16:14.555405+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.382 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[36207]: cluster 2023-12-07T17:16:14.646834+0000 mon.a (mon.0) 758 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:16:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.249367+0000 mgr.y (mgr.24626) 12 : cephadm 1 Adjusting osd_memory_target on smithi064 to 3804M 2023-12-07T17:16:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.644489+0000 mgr.y (mgr.24626) 13 : cephadm 1 Adjusting osd_memory_target on smithi184 to 3804M 2023-12-07T17:16:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.656569+0000 mgr.y (mgr.24626) 14 : cephadm 1 Updating smithi064:/etc/ceph/ceph.conf 2023-12-07T17:16:15.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.656959+0000 mgr.y (mgr.24626) 15 : cephadm 1 Updating smithi184:/etc/ceph/ceph.conf 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.821525+0000 mgr.y (mgr.24626) 16 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.972998+0000 mgr.y (mgr.24626) 17 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cluster 2023-12-07T17:16:13.974311+0000 mgr.y (mgr.24626) 18 : cluster 0 pgmap v7: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:13.986296+0000 mgr.y (mgr.24626) 19 : cephadm 1 Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:14.137742+0000 mgr.y (mgr.24626) 20 : cephadm 1 Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cephadm 2023-12-07T17:16:14.151976+0000 mgr.y (mgr.24626) 21 : cephadm 1 Updating smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: audit 2023-12-07T17:16:14.331819+0000 mon.a (mon.0) 753 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: audit 2023-12-07T17:16:14.344117+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: audit 2023-12-07T17:16:14.531905+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: audit 2023-12-07T17:16:14.544092+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: audit 2023-12-07T17:16:14.555405+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:15 smithi184 bash[31336]: cluster 2023-12-07T17:16:14.646834+0000 mon.a (mon.0) 758 : cluster 3 Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-12-07T17:16:15.617 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled grafana update... 2023-12-07T17:16:15.680 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:15 smithi064 systemd[1]: Started Ceph node-exporter.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:16:16.120 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:15 smithi064 bash[93122]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2023-12-07T17:16:16.369 DEBUG:teuthology.orchestra.run.smithi184:grafana.a> sudo journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@grafana.a.service 2023-12-07T17:16:16.370 INFO:tasks.cephadm:Setting up client nodes... 2023-12-07T17:16:16.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-07T17:16:16.479 INFO:journalctl@ceph.grafana.a.smithi184.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2023-12-07T17:16:16.559 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: cephadm 2023-12-07T17:16:14.302308+0000 mgr.y (mgr.24626) 22 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:16.559 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: cluster 2023-12-07T17:16:14.547157+0000 mgr.y (mgr.24626) 23 : cluster 0 pgmap v8: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2023-12-07T17:16:16.559 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: cephadm 2023-12-07T17:16:14.561804+0000 mgr.y (mgr.24626) 24 : cephadm 1 Deploying daemon node-exporter.a on smithi064 2023-12-07T17:16:16.559 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: audit 2023-12-07T17:16:15.602298+0000 mgr.y (mgr.24626) 25 : audit 0 from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:16.559 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: cephadm 2023-12-07T17:16:15.606819+0000 mgr.y (mgr.24626) 26 : cephadm 1 Saving service grafana spec with placement smithi184=a;count:1 2023-12-07T17:16:16.560 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: audit 2023-12-07T17:16:15.614111+0000 mon.a (mon.0) 759 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.560 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: audit 2023-12-07T17:16:15.673274+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.560 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: audit 2023-12-07T17:16:15.683607+0000 mon.a (mon.0) 761 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.560 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: audit 2023-12-07T17:16:15.735537+0000 mon.a (mon.0) 762 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.560 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[31336]: cephadm 2023-12-07T17:16:15.737591+0000 mgr.y (mgr.24626) 27 : cephadm 1 Deploying daemon node-exporter.b on smithi184 2023-12-07T17:16:16.619 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: v1.5.0: Pulling from prometheus/node-exporter 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: cephadm 2023-12-07T17:16:14.302308+0000 mgr.y (mgr.24626) 22 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: cluster 2023-12-07T17:16:14.547157+0000 mgr.y (mgr.24626) 23 : cluster 0 pgmap v8: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: cephadm 2023-12-07T17:16:14.561804+0000 mgr.y (mgr.24626) 24 : cephadm 1 Deploying daemon node-exporter.a on smithi064 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: audit 2023-12-07T17:16:15.602298+0000 mgr.y (mgr.24626) 25 : audit 0 from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: cephadm 2023-12-07T17:16:15.606819+0000 mgr.y (mgr.24626) 26 : cephadm 1 Saving service grafana spec with placement smithi184=a;count:1 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: audit 2023-12-07T17:16:15.614111+0000 mon.a (mon.0) 759 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: audit 2023-12-07T17:16:15.673274+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: audit 2023-12-07T17:16:15.683607+0000 mon.a (mon.0) 761 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: audit 2023-12-07T17:16:15.735537+0000 mon.a (mon.0) 762 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[36207]: cephadm 2023-12-07T17:16:15.737591+0000 mgr.y (mgr.24626) 27 : cephadm 1 Deploying daemon node-exporter.b on smithi184 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: cephadm 2023-12-07T17:16:14.302308+0000 mgr.y (mgr.24626) 22 : cephadm 1 Updating smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: cluster 2023-12-07T17:16:14.547157+0000 mgr.y (mgr.24626) 23 : cluster 0 pgmap v8: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: cephadm 2023-12-07T17:16:14.561804+0000 mgr.y (mgr.24626) 24 : cephadm 1 Deploying daemon node-exporter.a on smithi064 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: audit 2023-12-07T17:16:15.602298+0000 mgr.y (mgr.24626) 25 : audit 0 from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: cephadm 2023-12-07T17:16:15.606819+0000 mgr.y (mgr.24626) 26 : cephadm 1 Saving service grafana spec with placement smithi184=a;count:1 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: audit 2023-12-07T17:16:15.614111+0000 mon.a (mon.0) 759 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: audit 2023-12-07T17:16:15.673274+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: audit 2023-12-07T17:16:15.683607+0000 mon.a (mon.0) 761 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: audit 2023-12-07T17:16:15.735537+0000 mon.a (mon.0) 762 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:16.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[25056]: cephadm 2023-12-07T17:16:15.737591+0000 mgr.y (mgr.24626) 27 : cephadm 1 Deploying daemon node-exporter.b on smithi184 2023-12-07T17:16:17.019 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: 22b70bddd3ac: Pulling fs layer 2023-12-07T17:16:17.019 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: 5c12815fee55: Pulling fs layer 2023-12-07T17:16:17.019 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: c0e87333d380: Pulling fs layer 2023-12-07T17:16:17.019 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: 22b70bddd3ac: Download complete 2023-12-07T17:16:17.019 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:16 smithi064 bash[93122]: 5c12815fee55: Verifying Checksum 2023-12-07T17:16:17.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:17.318 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:16 smithi184 systemd[1]: Started Ceph node-exporter.b for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:16:17.318 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:16 smithi184 bash[81648]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2023-12-07T17:16:17.369 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[93122]: 22b70bddd3ac: Pull complete 2023-12-07T17:16:17.370 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[93122]: c0e87333d380: Verifying Checksum 2023-12-07T17:16:17.370 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[93122]: c0e87333d380: Download complete 2023-12-07T17:16:17.761 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: v1.5.0: Pulling from prometheus/node-exporter 2023-12-07T17:16:17.869 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[93122]: 5c12815fee55: Pull complete 2023-12-07T17:16:18.066 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: cluster 2023-12-07T17:16:16.547816+0000 mgr.y (mgr.24626) 28 : cluster 0 pgmap v9: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.878223+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.888743+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.910943+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.917733+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.930432+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: cephadm 2023-12-07T17:16:16.950291+0000 mgr.y (mgr.24626) 29 : cephadm 1 Deploying daemon alertmanager.a on smithi064 2023-12-07T17:16:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[31336]: audit 2023-12-07T17:16:16.990611+0000 mgr.y (mgr.24626) 30 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:18.067 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: 22b70bddd3ac: Pulling fs layer 2023-12-07T17:16:18.067 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: 5c12815fee55: Pulling fs layer 2023-12-07T17:16:18.067 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: c0e87333d380: Pulling fs layer 2023-12-07T17:16:18.067 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: 22b70bddd3ac: Verifying Checksum 2023-12-07T17:16:18.067 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:17 smithi184 bash[81648]: 22b70bddd3ac: Download complete 2023-12-07T17:16:18.068 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: c0e87333d380: Verifying Checksum 2023-12-07T17:16:18.068 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: c0e87333d380: Download complete 2023-12-07T17:16:18.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: cluster 2023-12-07T17:16:16.547816+0000 mgr.y (mgr.24626) 28 : cluster 0 pgmap v9: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:18.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.878223+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.888743+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.910943+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.917733+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.930432+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: cephadm 2023-12-07T17:16:16.950291+0000 mgr.y (mgr.24626) 29 : cephadm 1 Deploying daemon alertmanager.a on smithi064 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[25056]: audit 2023-12-07T17:16:16.990611+0000 mgr.y (mgr.24626) 30 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:18.166 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[93122]: c0e87333d380: Pull complete 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: cluster 2023-12-07T17:16:16.547816+0000 mgr.y (mgr.24626) 28 : cluster 0 pgmap v9: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.878223+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.166 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.888743+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.910943+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.917733+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.930432+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:18.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: cephadm 2023-12-07T17:16:16.950291+0000 mgr.y (mgr.24626) 29 : cephadm 1 Deploying daemon alertmanager.a on smithi064 2023-12-07T17:16:18.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:17 smithi064 bash[36207]: audit 2023-12-07T17:16:16.990611+0000 mgr.y (mgr.24626) 30 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:18.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: 5c12815fee55: Verifying Checksum 2023-12-07T17:16:18.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: 5c12815fee55: Download complete 2023-12-07T17:16:18.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: 22b70bddd3ac: Pull complete 2023-12-07T17:16:18.619 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:18 smithi064 bash[93122]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2023-12-07T17:16:18.620 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:18 smithi064 bash[93122]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2023-12-07T17:16:18.981 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: 5c12815fee55: Pull complete 2023-12-07T17:16:19.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:18 smithi184 bash[81648]: c0e87333d380: Pull complete 2023-12-07T17:16:19.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:19 smithi184 bash[81648]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2023-12-07T17:16:19.317 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:19 smithi184 bash[81648]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2023-12-07T17:16:20.175 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:19 smithi064 bash[36207]: cluster 2023-12-07T17:16:18.548729+0000 mgr.y (mgr.24626) 31 : cluster 0 pgmap v10: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:20.175 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:19 smithi064 bash[25056]: cluster 2023-12-07T17:16:18.548729+0000 mgr.y (mgr.24626) 31 : cluster 0 pgmap v10: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:20.304 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:19 smithi184 bash[31336]: cluster 2023-12-07T17:16:18.548729+0000 mgr.y (mgr.24626) 31 : cluster 0 pgmap v10: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.419Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.419Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.420Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2023-12-07T17:16:20.568 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=arp 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=bcache 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=bonding 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=btrfs 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=conntrack 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=cpu 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=cpufreq 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=diskstats 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=dmi 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=edac 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=entropy 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=fibrechannel 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=filefd 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=filesystem 2023-12-07T17:16:20.569 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=hwmon 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=infiniband 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=ipvs 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=loadavg 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=mdadm 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=meminfo 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=netclass 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=netdev 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=netstat 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=nfs 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=nfsd 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=nvme 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=os 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2023-12-07T17:16:20.570 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=pressure 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=rapl 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=schedstat 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=selinux 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=sockstat 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=softnet 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=stat 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=tapestats 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=textfile 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.421Z caller=node_exporter.go:117 level=info collector=thermal_zone 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=time 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=udp_queues 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=uname 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=vmstat 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=xfs 2023-12-07T17:16:20.571 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=node_exporter.go:117 level=info collector=zfs 2023-12-07T17:16:20.572 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2023-12-07T17:16:20.572 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:16:20 smithi184 bash[81648]: ts=2023-12-07T17:16:20.422Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2023-12-07T17:16:20.620 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.302Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2023-12-07T17:16:20.620 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.302Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2023-12-07T17:16:20.620 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.303Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.303Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.303Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.303Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=arp 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=bcache 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=bonding 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=btrfs 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=conntrack 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=cpu 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=cpufreq 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=diskstats 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=dmi 2023-12-07T17:16:20.621 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=edac 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=entropy 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=fibrechannel 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=filefd 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=filesystem 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=hwmon 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=infiniband 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=ipvs 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=loadavg 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=mdadm 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=meminfo 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=netclass 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=netdev 2023-12-07T17:16:20.622 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=netstat 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=nfs 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=nfsd 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=nvme 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=os 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=pressure 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=rapl 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=schedstat 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=selinux 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=sockstat 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=softnet 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=stat 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=tapestats 2023-12-07T17:16:20.623 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=textfile 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=thermal_zone 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=time 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=udp_queues 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=uname 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=vmstat 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=xfs 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.304Z caller=node_exporter.go:117 level=info collector=zfs 2023-12-07T17:16:20.625 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.305Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2023-12-07T17:16:20.626 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:16:20 smithi064 bash[93122]: ts=2023-12-07T17:16:20.305Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2023-12-07T17:16:22.137 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[25056]: cluster 2023-12-07T17:16:20.549334+0000 mgr.y (mgr.24626) 32 : cluster 0 pgmap v11: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[25056]: audit 2023-12-07T17:16:21.090470+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[25056]: audit 2023-12-07T17:16:21.148890+0000 mon.c (mon.2) 43 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[36207]: cluster 2023-12-07T17:16:20.549334+0000 mgr.y (mgr.24626) 32 : cluster 0 pgmap v11: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[36207]: audit 2023-12-07T17:16:21.090470+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:22.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:22 smithi064 bash[36207]: audit 2023-12-07T17:16:21.148890+0000 mon.c (mon.2) 43 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:22.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:22 smithi184 bash[31336]: cluster 2023-12-07T17:16:20.549334+0000 mgr.y (mgr.24626) 32 : cluster 0 pgmap v11: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-07T17:16:22.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:22 smithi184 bash[31336]: audit 2023-12-07T17:16:21.090470+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:22.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:22 smithi184 bash[31336]: audit 2023-12-07T17:16:21.148890+0000 mon.c (mon.2) 43 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:24.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:24 smithi064 bash[25056]: cluster 2023-12-07T17:16:22.550241+0000 mgr.y (mgr.24626) 33 : cluster 0 pgmap v12: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:24.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:24 smithi064 bash[36207]: cluster 2023-12-07T17:16:22.550241+0000 mgr.y (mgr.24626) 33 : cluster 0 pgmap v12: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:24.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:24 smithi184 bash[31336]: cluster 2023-12-07T17:16:22.550241+0000 mgr.y (mgr.24626) 33 : cluster 0 pgmap v12: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 12 op/s 2023-12-07T17:16:26.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:26 smithi064 bash[25056]: cluster 2023-12-07T17:16:24.550908+0000 mgr.y (mgr.24626) 34 : cluster 0 pgmap v13: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2023-12-07T17:16:26.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:26 smithi064 bash[36207]: cluster 2023-12-07T17:16:24.550908+0000 mgr.y (mgr.24626) 34 : cluster 0 pgmap v13: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2023-12-07T17:16:26.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:26 smithi184 bash[31336]: cluster 2023-12-07T17:16:24.550908+0000 mgr.y (mgr.24626) 34 : cluster 0 pgmap v13: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2023-12-07T17:16:27.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:26 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:27.921 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-12-07T17:16:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDr/XFlYFmENhAAH1PAPyzcvydAWpj6DPd+5Q== 2023-12-07T17:16:28.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[25056]: cluster 2023-12-07T17:16:26.551593+0000 mgr.y (mgr.24626) 35 : cluster 0 pgmap v14: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:28.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[25056]: audit 2023-12-07T17:16:26.994864+0000 mgr.y (mgr.24626) 36 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[25056]: audit 2023-12-07T17:16:27.914132+0000 mon.c (mon.2) 44 : audit 1 from='client.? 172.21.15.64:0/2169820585' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[25056]: audit 2023-12-07T17:16:27.914497+0000 mon.a (mon.0) 769 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[25056]: audit 2023-12-07T17:16:27.916485+0000 mon.a (mon.0) 770 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[36207]: cluster 2023-12-07T17:16:26.551593+0000 mgr.y (mgr.24626) 35 : cluster 0 pgmap v14: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[36207]: audit 2023-12-07T17:16:26.994864+0000 mgr.y (mgr.24626) 36 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[36207]: audit 2023-12-07T17:16:27.914132+0000 mon.c (mon.2) 44 : audit 1 from='client.? 172.21.15.64:0/2169820585' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[36207]: audit 2023-12-07T17:16:27.914497+0000 mon.a (mon.0) 769 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:28 smithi064 bash[36207]: audit 2023-12-07T17:16:27.916485+0000 mon.a (mon.0) 770 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:28.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:28 smithi184 bash[31336]: cluster 2023-12-07T17:16:26.551593+0000 mgr.y (mgr.24626) 35 : cluster 0 pgmap v14: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:28.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:28 smithi184 bash[31336]: audit 2023-12-07T17:16:26.994864+0000 mgr.y (mgr.24626) 36 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:28.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:28 smithi184 bash[31336]: audit 2023-12-07T17:16:27.914132+0000 mon.c (mon.2) 44 : audit 1 from='client.? 172.21.15.64:0/2169820585' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:28 smithi184 bash[31336]: audit 2023-12-07T17:16:27.914497+0000 mon.a (mon.0) 769 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:28.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:28 smithi184 bash[31336]: audit 2023-12-07T17:16:27.916485+0000 mon.a (mon.0) 770 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:29.120 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:16:29.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-07T17:16:29.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-07T17:16:29.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-07T17:16:29.609 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:29 smithi064 systemd[1]: Started Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: cluster 2023-12-07T17:16:28.552548+0000 mgr.y (mgr.24626) 37 : cluster 0 pgmap v15: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: audit 2023-12-07T17:16:29.592997+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: audit 2023-12-07T17:16:29.603646+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: audit 2023-12-07T17:16:29.647594+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: audit 2023-12-07T17:16:29.653069+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[25056]: audit 2023-12-07T17:16:29.718473+0000 mon.c (mon.2) 45 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: cluster 2023-12-07T17:16:28.552548+0000 mgr.y (mgr.24626) 37 : cluster 0 pgmap v15: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: audit 2023-12-07T17:16:29.592997+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: audit 2023-12-07T17:16:29.603646+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: audit 2023-12-07T17:16:29.647594+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: audit 2023-12-07T17:16:29.653069+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[36207]: audit 2023-12-07T17:16:29.718473+0000 mon.c (mon.2) 45 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: cluster 2023-12-07T17:16:28.552548+0000 mgr.y (mgr.24626) 37 : cluster 0 pgmap v15: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: audit 2023-12-07T17:16:29.592997+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: audit 2023-12-07T17:16:29.603646+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: audit 2023-12-07T17:16:29.647594+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: audit 2023-12-07T17:16:29.653069+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:30 smithi184 bash[31336]: audit 2023-12-07T17:16:29.718473+0000 mon.c (mon.2) 45 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.784Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.784Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.788Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.64 port=9094 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.789Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.835Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.836Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.841Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2023-12-07T17:16:31.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:30 smithi064 bash[93704]: ts=2023-12-07T17:16:30.841Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2023-12-07T17:16:31.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:31 smithi184 bash[31336]: audit 2023-12-07T17:16:30.250304+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:31 smithi184 bash[31336]: audit 2023-12-07T17:16:30.264372+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:31 smithi184 bash[31336]: cluster 2023-12-07T17:16:30.553238+0000 mgr.y (mgr.24626) 38 : cluster 0 pgmap v16: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:31.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:31 smithi184 bash[31336]: audit 2023-12-07T17:16:31.101624+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[25056]: audit 2023-12-07T17:16:30.250304+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[25056]: audit 2023-12-07T17:16:30.264372+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[25056]: cluster 2023-12-07T17:16:30.553238+0000 mgr.y (mgr.24626) 38 : cluster 0 pgmap v16: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[25056]: audit 2023-12-07T17:16:31.101624+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[36207]: audit 2023-12-07T17:16:30.250304+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[36207]: audit 2023-12-07T17:16:30.264372+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[36207]: cluster 2023-12-07T17:16:30.553238+0000 mgr.y (mgr.24626) 38 : cluster 0 pgmap v16: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:31.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:31 smithi064 bash[36207]: audit 2023-12-07T17:16:31.101624+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:33.120 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:32 smithi064 bash[93704]: ts=2023-12-07T17:16:32.790Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000149255s 2023-12-07T17:16:33.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:33 smithi064 bash[25056]: cluster 2023-12-07T17:16:32.554148+0000 mgr.y (mgr.24626) 39 : cluster 0 pgmap v17: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:33.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:33 smithi064 bash[36207]: cluster 2023-12-07T17:16:32.554148+0000 mgr.y (mgr.24626) 39 : cluster 0 pgmap v17: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:33.919 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T17:16:33.948 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:33 smithi184 bash[31336]: cluster 2023-12-07T17:16:32.554148+0000 mgr.y (mgr.24626) 39 : cluster 0 pgmap v17: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:35.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:35 smithi064 bash[25056]: cluster 2023-12-07T17:16:34.554789+0000 mgr.y (mgr.24626) 40 : cluster 0 pgmap v18: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:35.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:35 smithi064 bash[36207]: cluster 2023-12-07T17:16:34.554789+0000 mgr.y (mgr.24626) 40 : cluster 0 pgmap v18: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:36.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:35 smithi184 bash[31336]: cluster 2023-12-07T17:16:34.554789+0000 mgr.y (mgr.24626) 40 : cluster 0 pgmap v18: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE Bus STOPPING 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE Bus STOPPED 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE Bus STARTING 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE Serving on http://:::9283 2023-12-07T17:16:36.622 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25333]: [07/Dec/2023:17:16:36] ENGINE Bus STARTED 2023-12-07T17:16:37.006 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:35.816363+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.006 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:35.829699+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.007 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.008932+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.007 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.022466+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.007 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.025623+0000 mon.c (mon.2) 46 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:37.007 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.027217+0000 mon.c (mon.2) 47 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:37.007 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: cluster 2023-12-07T17:16:36.029759+0000 mgr.y (mgr.24626) 41 : cluster 0 pgmap v19: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 0 op/s 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: cluster 2023-12-07T17:16:36.030216+0000 mgr.y (mgr.24626) 42 : cluster 0 pgmap v20: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.035467+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: cephadm 2023-12-07T17:16:36.058861+0000 mgr.y (mgr.24626) 43 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.083179+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.099336+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.106356+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.107025+0000 mgr.y (mgr.24626) 44 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.112856+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: cephadm 2023-12-07T17:16:36.126975+0000 mgr.y (mgr.24626) 45 : cephadm 1 Deploying daemon grafana.a on smithi184 2023-12-07T17:16:37.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:36 smithi184 bash[31336]: audit 2023-12-07T17:16:36.166568+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:37.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:35.816363+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:35.829699+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.008932+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.022466+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.025623+0000 mon.c (mon.2) 46 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.027217+0000 mon.c (mon.2) 47 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: cluster 2023-12-07T17:16:36.029759+0000 mgr.y (mgr.24626) 41 : cluster 0 pgmap v19: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 0 op/s 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: cluster 2023-12-07T17:16:36.030216+0000 mgr.y (mgr.24626) 42 : cluster 0 pgmap v20: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.035467+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: cephadm 2023-12-07T17:16:36.058861+0000 mgr.y (mgr.24626) 43 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.083179+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.099336+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.106356+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.107025+0000 mgr.y (mgr.24626) 44 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.112856+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: cephadm 2023-12-07T17:16:36.126975+0000 mgr.y (mgr.24626) 45 : cephadm 1 Deploying daemon grafana.a on smithi184 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[25056]: audit 2023-12-07T17:16:36.166568+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:35.816363+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:35.829699+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.008932+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.022466+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.025623+0000 mon.c (mon.2) 46 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.027217+0000 mon.c (mon.2) 47 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: cluster 2023-12-07T17:16:36.029759+0000 mgr.y (mgr.24626) 41 : cluster 0 pgmap v19: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 0 op/s 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: cluster 2023-12-07T17:16:36.030216+0000 mgr.y (mgr.24626) 42 : cluster 0 pgmap v20: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.035467+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: cephadm 2023-12-07T17:16:36.058861+0000 mgr.y (mgr.24626) 43 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.083179+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.099336+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.106356+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.107025+0000 mgr.y (mgr.24626) 44 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.112856+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: cephadm 2023-12-07T17:16:36.126975+0000 mgr.y (mgr.24626) 45 : cephadm 1 Deploying daemon grafana.a on smithi184 2023-12-07T17:16:37.123 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:36 smithi064 bash[36207]: audit 2023-12-07T17:16:36.166568+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:37.144 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2023-12-07T17:16:37.144 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQD1/XFlJ9cMCBAAaGL52L60MKwLXYiZ4/+i5A== 2023-12-07T17:16:37.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:38.006 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:16:38.006 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2023-12-07T17:16:38.006 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-07T17:16:38.031 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-07T17:16:38.031 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-07T17:16:38.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-12-07T17:16:38.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:38 smithi184 bash[31336]: audit 2023-12-07T17:16:37.003877+0000 mgr.y (mgr.24626) 46 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:38.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:38 smithi184 bash[31336]: audit 2023-12-07T17:16:37.133629+0000 mon.b (mon.1) 33 : audit 1 from='client.? 172.21.15.184:0/4012410319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:38 smithi184 bash[31336]: audit 2023-12-07T17:16:37.134618+0000 mon.a (mon.0) 786 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:38 smithi184 bash[31336]: audit 2023-12-07T17:16:37.138471+0000 mon.a (mon.0) 787 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[25056]: audit 2023-12-07T17:16:37.003877+0000 mgr.y (mgr.24626) 46 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[25056]: audit 2023-12-07T17:16:37.133629+0000 mon.b (mon.1) 33 : audit 1 from='client.? 172.21.15.184:0/4012410319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[25056]: audit 2023-12-07T17:16:37.134618+0000 mon.a (mon.0) 786 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[25056]: audit 2023-12-07T17:16:37.138471+0000 mon.a (mon.0) 787 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[36207]: audit 2023-12-07T17:16:37.003877+0000 mgr.y (mgr.24626) 46 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[36207]: audit 2023-12-07T17:16:37.133629+0000 mon.b (mon.1) 33 : audit 1 from='client.? 172.21.15.184:0/4012410319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[36207]: audit 2023-12-07T17:16:37.134618+0000 mon.a (mon.0) 786 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T17:16:38.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:38 smithi064 bash[36207]: audit 2023-12-07T17:16:37.138471+0000 mon.a (mon.0) 787 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T17:16:39.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:39 smithi184 bash[31336]: cluster 2023-12-07T17:16:38.030998+0000 mgr.y (mgr.24626) 47 : cluster 0 pgmap v21: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:39.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:39 smithi064 bash[25056]: cluster 2023-12-07T17:16:38.030998+0000 mgr.y (mgr.24626) 47 : cluster 0 pgmap v21: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:39.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:39 smithi064 bash[36207]: cluster 2023-12-07T17:16:38.030998+0000 mgr.y (mgr.24626) 47 : cluster 0 pgmap v21: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:41.090 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:16:40 smithi064 bash[93704]: ts=2023-12-07T17:16:40.792Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002713845s 2023-12-07T17:16:41.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:41 smithi064 bash[25056]: cluster 2023-12-07T17:16:40.031632+0000 mgr.y (mgr.24626) 48 : cluster 0 pgmap v22: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:41.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:41 smithi064 bash[36207]: cluster 2023-12-07T17:16:40.031632+0000 mgr.y (mgr.24626) 48 : cluster 0 pgmap v22: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:41.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:41 smithi184 bash[31336]: cluster 2023-12-07T17:16:40.031632+0000 mgr.y (mgr.24626) 48 : cluster 0 pgmap v22: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:16:43.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:43 smithi064 bash[25056]: cluster 2023-12-07T17:16:42.033174+0000 mgr.y (mgr.24626) 49 : cluster 0 pgmap v23: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:43.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:43 smithi064 bash[36207]: cluster 2023-12-07T17:16:42.033174+0000 mgr.y (mgr.24626) 49 : cluster 0 pgmap v23: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:43.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:43 smithi184 bash[31336]: cluster 2023-12-07T17:16:42.033174+0000 mgr.y (mgr.24626) 49 : cluster 0 pgmap v23: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:45.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:45 smithi064 bash[25056]: cluster 2023-12-07T17:16:44.033819+0000 mgr.y (mgr.24626) 50 : cluster 0 pgmap v24: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:45.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:45 smithi064 bash[36207]: cluster 2023-12-07T17:16:44.033819+0000 mgr.y (mgr.24626) 50 : cluster 0 pgmap v24: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:45.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:45 smithi184 bash[31336]: cluster 2023-12-07T17:16:44.033819+0000 mgr.y (mgr.24626) 50 : cluster 0 pgmap v24: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:16:46.462 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:16:46.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:46 smithi064 bash[25056]: cluster 2023-12-07T17:16:46.034463+0000 mgr.y (mgr.24626) 51 : cluster 0 pgmap v25: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:16:46.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:46 smithi064 bash[25056]: audit 2023-12-07T17:16:46.448983+0000 mon.c (mon.2) 50 : audit 0 from='client.? 172.21.15.64:0/673271688' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T17:16:46.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:46 smithi064 bash[36207]: cluster 2023-12-07T17:16:46.034463+0000 mgr.y (mgr.24626) 51 : cluster 0 pgmap v25: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:16:46.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:46 smithi064 bash[36207]: audit 2023-12-07T17:16:46.448983+0000 mon.c (mon.2) 50 : audit 0 from='client.? 172.21.15.64:0/673271688' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T17:16:47.017 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:46 smithi184 bash[31336]: cluster 2023-12-07T17:16:46.034463+0000 mgr.y (mgr.24626) 51 : cluster 0 pgmap v25: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:16:47.017 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:46 smithi184 bash[31336]: audit 2023-12-07T17:16:46.448983+0000 mon.c (mon.2) 50 : audit 0 from='client.? 172.21.15.64:0/673271688' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T17:16:47.206 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"active_gid":24626,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":357132384},{"type":"v1","addr":"172.21.15.64:6801","nonce":357132384}]},"active_addr":"172.21.15.64:6801/357132384","active_change":"2023-12-07T17:16:05.952215+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24610,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/","prometheus":"http://172.21.15.64:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":71,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2128610952}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2061628533}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1947472826}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2499076320}]}]} 2023-12-07T17:16:47.210 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-07T17:16:47.210 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-07T17:16:47.210 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T17:16:47.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:48.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:47 smithi184 bash[31336]: audit 2023-12-07T17:16:47.014758+0000 mgr.y (mgr.24626) 52 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:48.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:47 smithi064 bash[25056]: audit 2023-12-07T17:16:47.014758+0000 mgr.y (mgr.24626) 52 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:48.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:47 smithi064 bash[36207]: audit 2023-12-07T17:16:47.014758+0000 mgr.y (mgr.24626) 52 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:49.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:48 smithi184 bash[31336]: cluster 2023-12-07T17:16:48.035396+0000 mgr.y (mgr.24626) 53 : cluster 0 pgmap v26: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:49.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:48 smithi064 bash[25056]: cluster 2023-12-07T17:16:48.035396+0000 mgr.y (mgr.24626) 53 : cluster 0 pgmap v26: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:49.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:48 smithi064 bash[36207]: cluster 2023-12-07T17:16:48.035396+0000 mgr.y (mgr.24626) 53 : cluster 0 pgmap v26: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:51.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:51 smithi064 bash[25056]: cluster 2023-12-07T17:16:50.036061+0000 mgr.y (mgr.24626) 54 : cluster 0 pgmap v27: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:51.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:51 smithi064 bash[36207]: cluster 2023-12-07T17:16:50.036061+0000 mgr.y (mgr.24626) 54 : cluster 0 pgmap v27: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:51.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:51 smithi184 bash[31336]: cluster 2023-12-07T17:16:50.036061+0000 mgr.y (mgr.24626) 54 : cluster 0 pgmap v27: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:51.988 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:16:52.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:52 smithi064 bash[25056]: audit 2023-12-07T17:16:51.188094+0000 mon.c (mon.2) 51 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:52.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:52 smithi064 bash[36207]: audit 2023-12-07T17:16:51.188094+0000 mon.c (mon.2) 51 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:52.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:52 smithi184 bash[31336]: audit 2023-12-07T17:16:51.188094+0000 mon.c (mon.2) 51 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:16:53.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:53 smithi064 bash[25056]: cluster 2023-12-07T17:16:52.036846+0000 mgr.y (mgr.24626) 55 : cluster 0 pgmap v28: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:53.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:53 smithi064 bash[36207]: cluster 2023-12-07T17:16:52.036846+0000 mgr.y (mgr.24626) 55 : cluster 0 pgmap v28: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:53.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:53 smithi184 bash[31336]: cluster 2023-12-07T17:16:52.036846+0000 mgr.y (mgr.24626) 55 : cluster 0 pgmap v28: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:55 smithi064 bash[25056]: cluster 2023-12-07T17:16:54.037471+0000 mgr.y (mgr.24626) 56 : cluster 0 pgmap v29: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:55.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:55 smithi064 bash[36207]: cluster 2023-12-07T17:16:54.037471+0000 mgr.y (mgr.24626) 56 : cluster 0 pgmap v29: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:55.487 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:16:55.487 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":71,"fsid":"7ae18f76-9522-11ee-95a2-87774f69a715","created":"2023-12-07T17:04:13.885658+0000","modified":"2023-12-07T17:16:05.951943+0000","last_up_change":"2023-12-07T17:14:28.682548+0000","last_in_change":"2023-12-07T17:14:00.148477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T17:09:56.938253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2023-12-07T17:15:08.623846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2023-12-07T17:15:12.144235+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":67,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beb84ad8-fb61-45c8-99fa-eb8a522bcca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6803","nonce":4255677702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6805","nonce":4255677702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6809","nonce":4255677702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6807","nonce":4255677702}]},"public_addr":"172.21.15.64:6803/4255677702","cluster_addr":"172.21.15.64:6805/4255677702","heartbeat_back_addr":"172.21.15.64:6809/4255677702","heartbeat_front_addr":"172.21.15.64:6807/4255677702","state":["exists","up"]},{"osd":1,"uuid":"5fda1ac8-8149-447f-ab1f-5c6192928f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6811","nonce":1129839528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6813","nonce":1129839528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6817","nonce":1129839528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6815","nonce":1129839528}]},"public_addr":"172.21.15.64:6811/1129839528","cluster_addr":"172.21.15.64:6813/1129839528","heartbeat_back_addr":"172.21.15.64:6817/1129839528","heartbeat_front_addr":"172.21.15.64:6815/1129839528","state":["exists","up"]},{"osd":2,"uuid":"36cb5cff-a8e8-46cb-86c5-55243883da50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6819","nonce":135846840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6821","nonce":135846840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6825","nonce":135846840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6823","nonce":135846840}]},"public_addr":"172.21.15.64:6819/135846840","cluster_addr":"172.21.15.64:6821/135846840","heartbeat_back_addr":"172.21.15.64:6825/135846840","heartbeat_front_addr":"172.21.15.64:6823/135846840","state":["exists","up"]},{"osd":3,"uuid":"22c06021-9405-4ee7-8b82-1b3ee1a7b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6827","nonce":2613043191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6829","nonce":2613043191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6833","nonce":2613043191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6831","nonce":2613043191}]},"public_addr":"172.21.15.64:6827/2613043191","cluster_addr":"172.21.15.64:6829/2613043191","heartbeat_back_addr":"172.21.15.64:6833/2613043191","heartbeat_front_addr":"172.21.15.64:6831/2613043191","state":["exists","up"]},{"osd":4,"uuid":"9be6b07a-5939-4ac2-abaa-f889afc87cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6801","nonce":2660335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6803","nonce":2660335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6807","nonce":2660335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6805","nonce":2660335559}]},"public_addr":"172.21.15.184:6801/2660335559","cluster_addr":"172.21.15.184:6803/2660335559","heartbeat_back_addr":"172.21.15.184:6807/2660335559","heartbeat_front_addr":"172.21.15.184:6805/2660335559","state":["exists","up"]},{"osd":5,"uuid":"2df3256b-b1c8-4b26-82c6-55d945e2506c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6809","nonce":911723058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6811","nonce":911723058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6815","nonce":911723058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6813","nonce":911723058}]},"public_addr":"172.21.15.184:6809/911723058","cluster_addr":"172.21.15.184:6811/911723058","heartbeat_back_addr":"172.21.15.184:6815/911723058","heartbeat_front_addr":"172.21.15.184:6813/911723058","state":["exists","up"]},{"osd":6,"uuid":"0ae74eb7-8166-414d-abc5-7f0ecb5f4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6817","nonce":3929847913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6819","nonce":3929847913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6823","nonce":3929847913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6821","nonce":3929847913}]},"public_addr":"172.21.15.184:6817/3929847913","cluster_addr":"172.21.15.184:6819/3929847913","heartbeat_back_addr":"172.21.15.184:6823/3929847913","heartbeat_front_addr":"172.21.15.184:6821/3929847913","state":["exists","up"]},{"osd":7,"uuid":"a1498aa6-78dd-4a46-9a67-4454cb6f66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6825","nonce":648979552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6827","nonce":648979552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6831","nonce":648979552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6829","nonce":648979552}]},"public_addr":"172.21.15.184:6825/648979552","cluster_addr":"172.21.15.184:6827/648979552","heartbeat_back_addr":"172.21.15.184:6831/648979552","heartbeat_front_addr":"172.21.15.184:6829/648979552","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:07:59.780610+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:08:55.703708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:09:53.615215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:10:50.468115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:11:43.718719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:12:37.349365+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:13:32.110469+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:14:26.660719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/777432207":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6800/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/1799325112":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/539044856":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6801/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/2476993930":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/1274328280":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/43744120":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/587612034":"2023-12-08T17:04:48.222204+0000","172.21.15.64:6800/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:6800/777432207":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/4265627278":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6801/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/2277610507":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/3925146267":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/2213461460":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/360843810":"2023-12-08T17:16:05.951874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T17:16:55.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:55 smithi184 bash[31336]: cluster 2023-12-07T17:16:54.037471+0000 mgr.y (mgr.24626) 56 : cluster 0 pgmap v29: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:16:56.302 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-07T17:16:56.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T17:16:56.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:56 smithi064 bash[25056]: audit 2023-12-07T17:16:55.483951+0000 mon.c (mon.2) 52 : audit 0 from='client.? 172.21.15.64:0/4125272938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:16:56.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:56 smithi064 bash[36207]: audit 2023-12-07T17:16:55.483951+0000 mon.c (mon.2) 52 : audit 0 from='client.? 172.21.15.64:0/4125272938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:16:56.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:56 smithi184 bash[31336]: audit 2023-12-07T17:16:55.483951+0000 mon.c (mon.2) 52 : audit 0 from='client.? 172.21.15.64:0/4125272938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:16:57.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:16:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:16:57.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:57 smithi184 bash[31336]: cluster 2023-12-07T17:16:56.038163+0000 mgr.y (mgr.24626) 57 : cluster 0 pgmap v30: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:57.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:57 smithi064 bash[25056]: cluster 2023-12-07T17:16:56.038163+0000 mgr.y (mgr.24626) 57 : cluster 0 pgmap v30: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:57.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:57 smithi064 bash[36207]: cluster 2023-12-07T17:16:56.038163+0000 mgr.y (mgr.24626) 57 : cluster 0 pgmap v30: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:16:58.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:58 smithi184 bash[31336]: audit 2023-12-07T17:16:57.025949+0000 mgr.y (mgr.24626) 58 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:58.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:58 smithi064 bash[25056]: audit 2023-12-07T17:16:57.025949+0000 mgr.y (mgr.24626) 58 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:58.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:58 smithi064 bash[36207]: audit 2023-12-07T17:16:57.025949+0000 mgr.y (mgr.24626) 58 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:16:59.415 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:16:59 smithi184 bash[31336]: cluster 2023-12-07T17:16:58.039076+0000 mgr.y (mgr.24626) 59 : cluster 0 pgmap v31: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:59.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:16:59 smithi064 bash[25056]: cluster 2023-12-07T17:16:58.039076+0000 mgr.y (mgr.24626) 59 : cluster 0 pgmap v31: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:16:59.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:16:59 smithi064 bash[36207]: cluster 2023-12-07T17:16:58.039076+0000 mgr.y (mgr.24626) 59 : cluster 0 pgmap v31: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:00.738 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:00 smithi184 systemd[1]: Started Ceph grafana.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:17:01.082 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: cluster 2023-12-07T17:17:00.039810+0000 mgr.y (mgr.24626) 60 : cluster 0 pgmap v32: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:00.727824+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:00.741697+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:00.768232+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:00.775698+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:00.841594+0000 mon.c (mon.2) 53 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[25056]: audit 2023-12-07T17:17:01.114626+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: cluster 2023-12-07T17:17:00.039810+0000 mgr.y (mgr.24626) 60 : cluster 0 pgmap v32: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:00.727824+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:00.741697+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:00.768232+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:00.775698+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:00.841594+0000 mon.c (mon.2) 53 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:01.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:01 smithi064 bash[36207]: audit 2023-12-07T17:17:01.114626+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.547 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: cluster 2023-12-07T17:17:00.039810+0000 mgr.y (mgr.24626) 60 : cluster 0 pgmap v32: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:01.547 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:00.727824+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.547 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:00.741697+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.547 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:00.768232+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.548 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:00.775698+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:01.548 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:00.841594+0000 mon.c (mon.2) 53 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:01.548 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[31336]: audit 2023-12-07T17:17:01.114626+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:02.169 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881588325Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881784295Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881806512Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881814705Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881821905Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881829741Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.88183683Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881844418Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881863111Z level=info msg="Path Home" path=/usr/share/grafana 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881876377Z level=info msg="Path Data" path=/var/lib/grafana 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881883225Z level=info msg="Path Logs" path=/var/log/grafana 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881893339Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2023-12-07T17:17:02.170 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881905309Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=settings t=2023-12-07T17:17:01.881917142Z level=info msg="App mode production" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=sqlstore t=2023-12-07T17:17:01.881985806Z level=info msg="Connecting to DB" dbtype=sqlite3 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=sqlstore t=2023-12-07T17:17:01.882007782Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:01.882987506Z level=info msg="Starting DB migrations" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:01.883087796Z level=info msg="Executing migration" id="create migration_log table" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:01.931790386Z level=info msg="Executing migration" id="create user table" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:01.957013876Z level=info msg="Executing migration" id="add unique index user.login" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:01 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:01.982345625Z level=info msg="Executing migration" id="add unique index user.email" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.032993244Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.07484435Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.100177417Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.125439221Z level=info msg="Executing migration" id="create user table v2" 2023-12-07T17:17:02.171 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.167335773Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.209342175Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.234624794Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.259884281Z level=info msg="Executing migration" id="Drop old table user_v1" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.285397386Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.327301781Z level=info msg="Executing migration" id="Update user table charset" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.369576421Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.411616449Z level=info msg="Executing migration" id="Add missing user data" 2023-12-07T17:17:02.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.461912036Z level=info msg="Executing migration" id="Add is_disabled column to user" 2023-12-07T17:17:02.724 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.495616102Z level=info msg="Executing migration" id="Add index user.login/user.email" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.537590305Z level=info msg="Executing migration" id="Add is_service_account column to user" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.562850026Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.604876022Z level=info msg="Executing migration" id="create temp user table v1-7" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.630171379Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.672193441Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.697440123Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2023-12-07T17:17:02.725 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.72272248Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2023-12-07T17:17:02.993 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.747929928Z level=info msg="Executing migration" id="Update temp_user table charset" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.773118157Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.798365943Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.823509133Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.84869532Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.873999784Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.899194343Z level=info msg="Executing migration" id="create temp_user v2" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.941243054Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.966459453Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2023-12-07T17:17:02.994 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:02 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:02.991635709Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2023-12-07T17:17:03.254 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[31336]: cluster 2023-12-07T17:17:02.040828+0000 mgr.y (mgr.24626) 61 : cluster 0 pgmap v33: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.033868304Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.075837288Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.117735112Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.15980007Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.185040479Z level=info msg="Executing migration" id="create star table" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.227240505Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2023-12-07T17:17:03.255 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.252903745Z level=info msg="Executing migration" id="create org table v1" 2023-12-07T17:17:03.515 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.277951797Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2023-12-07T17:17:03.515 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.303267111Z level=info msg="Executing migration" id="create org_user table v1" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.328415337Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.370417621Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.412423207Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.437954453Z level=info msg="Executing migration" id="Update org table charset" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.462970189Z level=info msg="Executing migration" id="Update org_user table charset" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.488320771Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2023-12-07T17:17:03.516 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.513574515Z level=info msg="Executing migration" id="create dashboard table" 2023-12-07T17:17:03.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:03 smithi064 bash[25056]: cluster 2023-12-07T17:17:02.040828+0000 mgr.y (mgr.24626) 61 : cluster 0 pgmap v33: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:03.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:03 smithi064 bash[36207]: cluster 2023-12-07T17:17:02.040828+0000 mgr.y (mgr.24626) 61 : cluster 0 pgmap v33: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:03.776 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.555640913Z level=info msg="Executing migration" id="add index dashboard.account_id" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.580954853Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.622926023Z level=info msg="Executing migration" id="create dashboard_tag table" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.648232002Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.673537869Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.698752974Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.724027275Z level=info msg="Executing migration" id="create dashboard v2" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.749247193Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2023-12-07T17:17:03.777 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.774598422Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2023-12-07T17:17:04.028 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.799816917Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2023-12-07T17:17:04.028 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.824788463Z level=info msg="Executing migration" id="drop table dashboard_v1" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.850302013Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.875418267Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.917403524Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.942728925Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:03 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:03.984768536Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2023-12-07T17:17:04.029 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.026727011Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2023-12-07T17:17:04.298 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.051975757Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2023-12-07T17:17:04.298 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.077241311Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2023-12-07T17:17:04.298 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.136068168Z level=info msg="Executing migration" id="Update dashboard table charset" 2023-12-07T17:17:04.298 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.178002945Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2023-12-07T17:17:04.301 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.220171098Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2023-12-07T17:17:04.301 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.245604485Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2023-12-07T17:17:04.301 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.270842782Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2023-12-07T17:17:04.301 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.296077865Z level=info msg="Executing migration" id="Add column uid in dashboard" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.321300405Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.363165041Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.38860145Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.430505576Z level=info msg="Executing migration" id="Update dashboard title length" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.455521243Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.480881045Z level=info msg="Executing migration" id="create dashboard_provisioning" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.522980217Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2023-12-07T17:17:04.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.564916065Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2023-12-07T17:17:04.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:17:04.766 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":71,"fsid":"7ae18f76-9522-11ee-95a2-87774f69a715","created":"2023-12-07T17:04:13.885658+0000","modified":"2023-12-07T17:16:05.951943+0000","last_up_change":"2023-12-07T17:14:28.682548+0000","last_in_change":"2023-12-07T17:14:00.148477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T17:09:56.938253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2023-12-07T17:15:08.623846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2023-12-07T17:15:12.144235+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":67,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"beb84ad8-fb61-45c8-99fa-eb8a522bcca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6803","nonce":4255677702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6805","nonce":4255677702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6809","nonce":4255677702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":4255677702},{"type":"v1","addr":"172.21.15.64:6807","nonce":4255677702}]},"public_addr":"172.21.15.64:6803/4255677702","cluster_addr":"172.21.15.64:6805/4255677702","heartbeat_back_addr":"172.21.15.64:6809/4255677702","heartbeat_front_addr":"172.21.15.64:6807/4255677702","state":["exists","up"]},{"osd":1,"uuid":"5fda1ac8-8149-447f-ab1f-5c6192928f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6811","nonce":1129839528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6813","nonce":1129839528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6817","nonce":1129839528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1129839528},{"type":"v1","addr":"172.21.15.64:6815","nonce":1129839528}]},"public_addr":"172.21.15.64:6811/1129839528","cluster_addr":"172.21.15.64:6813/1129839528","heartbeat_back_addr":"172.21.15.64:6817/1129839528","heartbeat_front_addr":"172.21.15.64:6815/1129839528","state":["exists","up"]},{"osd":2,"uuid":"36cb5cff-a8e8-46cb-86c5-55243883da50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6819","nonce":135846840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6821","nonce":135846840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6825","nonce":135846840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":135846840},{"type":"v1","addr":"172.21.15.64:6823","nonce":135846840}]},"public_addr":"172.21.15.64:6819/135846840","cluster_addr":"172.21.15.64:6821/135846840","heartbeat_back_addr":"172.21.15.64:6825/135846840","heartbeat_front_addr":"172.21.15.64:6823/135846840","state":["exists","up"]},{"osd":3,"uuid":"22c06021-9405-4ee7-8b82-1b3ee1a7b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6827","nonce":2613043191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6829","nonce":2613043191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6833","nonce":2613043191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2613043191},{"type":"v1","addr":"172.21.15.64:6831","nonce":2613043191}]},"public_addr":"172.21.15.64:6827/2613043191","cluster_addr":"172.21.15.64:6829/2613043191","heartbeat_back_addr":"172.21.15.64:6833/2613043191","heartbeat_front_addr":"172.21.15.64:6831/2613043191","state":["exists","up"]},{"osd":4,"uuid":"9be6b07a-5939-4ac2-abaa-f889afc87cdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6801","nonce":2660335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6803","nonce":2660335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6807","nonce":2660335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2660335559},{"type":"v1","addr":"172.21.15.184:6805","nonce":2660335559}]},"public_addr":"172.21.15.184:6801/2660335559","cluster_addr":"172.21.15.184:6803/2660335559","heartbeat_back_addr":"172.21.15.184:6807/2660335559","heartbeat_front_addr":"172.21.15.184:6805/2660335559","state":["exists","up"]},{"osd":5,"uuid":"2df3256b-b1c8-4b26-82c6-55d945e2506c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6809","nonce":911723058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6811","nonce":911723058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6815","nonce":911723058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":911723058},{"type":"v1","addr":"172.21.15.184:6813","nonce":911723058}]},"public_addr":"172.21.15.184:6809/911723058","cluster_addr":"172.21.15.184:6811/911723058","heartbeat_back_addr":"172.21.15.184:6815/911723058","heartbeat_front_addr":"172.21.15.184:6813/911723058","state":["exists","up"]},{"osd":6,"uuid":"0ae74eb7-8166-414d-abc5-7f0ecb5f4fce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6817","nonce":3929847913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6819","nonce":3929847913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6823","nonce":3929847913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3929847913},{"type":"v1","addr":"172.21.15.184:6821","nonce":3929847913}]},"public_addr":"172.21.15.184:6817/3929847913","cluster_addr":"172.21.15.184:6819/3929847913","heartbeat_back_addr":"172.21.15.184:6823/3929847913","heartbeat_front_addr":"172.21.15.184:6821/3929847913","state":["exists","up"]},{"osd":7,"uuid":"a1498aa6-78dd-4a46-9a67-4454cb6f66fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6825","nonce":648979552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6827","nonce":648979552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6831","nonce":648979552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":648979552},{"type":"v1","addr":"172.21.15.184:6829","nonce":648979552}]},"public_addr":"172.21.15.184:6825/648979552","cluster_addr":"172.21.15.184:6827/648979552","heartbeat_back_addr":"172.21.15.184:6831/648979552","heartbeat_front_addr":"172.21.15.184:6829/648979552","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:07:59.780610+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:08:55.703708+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:09:53.615215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:10:50.468115+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:11:43.718719+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:12:37.349365+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:13:32.110469+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T17:14:26.660719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/777432207":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6800/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/1799325112":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/539044856":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6801/684854980":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/2476993930":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/1274328280":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/43744120":"2023-12-08T17:04:48.222204+0000","172.21.15.64:0/587612034":"2023-12-08T17:04:48.222204+0000","172.21.15.64:6800/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:6800/777432207":"2023-12-08T17:16:05.951874+0000","172.21.15.64:0/4265627278":"2023-12-08T17:16:05.951874+0000","172.21.15.64:6801/4024545334":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/2277610507":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/3925146267":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/2213461460":"2023-12-08T17:05:26.278541+0000","172.21.15.64:0/360843810":"2023-12-08T17:16:05.951874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T17:17:04.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.59022017Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2023-12-07T17:17:04.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.632217843Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2023-12-07T17:17:04.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.674232969Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2023-12-07T17:17:04.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.699475594Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2023-12-07T17:17:04.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.724601174Z level=info msg="Executing migration" id="Add check_sum column" 2023-12-07T17:17:04.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.749919616Z level=info msg="Executing migration" id="Add index for dashboard_title" 2023-12-07T17:17:04.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.791902699Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2023-12-07T17:17:05.097 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.833896285Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2023-12-07T17:17:05.097 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.859163491Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2023-12-07T17:17:05.097 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.884493418Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2023-12-07T17:17:05.097 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.909782899Z level=info msg="Executing migration" id="create data_source table" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.951663584Z level=info msg="Executing migration" id="add index data_source.account_id" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:04 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:04.993882825Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.019365439Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.044566271Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.069883053Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2023-12-07T17:17:05.098 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.095062049Z level=info msg="Executing migration" id="create data_source table v2" 2023-12-07T17:17:05.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:05 smithi064 bash[25056]: cluster 2023-12-07T17:17:04.041545+0000 mgr.y (mgr.24626) 62 : cluster 0 pgmap v34: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:05.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:05 smithi064 bash[25056]: audit 2023-12-07T17:17:04.760730+0000 mon.a (mon.0) 793 : audit 0 from='client.? 172.21.15.64:0/3271111533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:17:05.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:05 smithi064 bash[36207]: cluster 2023-12-07T17:17:04.041545+0000 mgr.y (mgr.24626) 62 : cluster 0 pgmap v34: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:05.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:05 smithi064 bash[36207]: audit 2023-12-07T17:17:04.760730+0000 mon.a (mon.0) 793 : audit 0 from='client.? 172.21.15.64:0/3271111533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:17:05.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[31336]: cluster 2023-12-07T17:17:04.041545+0000 mgr.y (mgr.24626) 62 : cluster 0 pgmap v34: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:05.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[31336]: audit 2023-12-07T17:17:04.760730+0000 mon.a (mon.0) 793 : audit 0 from='client.? 172.21.15.64:0/3271111533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T17:17:05.374 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.120386091Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2023-12-07T17:17:05.374 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.162498367Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2023-12-07T17:17:05.374 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.187686692Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2023-12-07T17:17:05.375 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.229665373Z level=info msg="Executing migration" id="Add column with_credentials" 2023-12-07T17:17:05.375 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.254735419Z level=info msg="Executing migration" id="Add secure json data column" 2023-12-07T17:17:05.375 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.279943249Z level=info msg="Executing migration" id="Update data_source table charset" 2023-12-07T17:17:05.375 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.3052741Z level=info msg="Executing migration" id="Update initial version to 1" 2023-12-07T17:17:05.375 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.372447177Z level=info msg="Executing migration" id="Add read_only data column" 2023-12-07T17:17:05.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-07T17:17:05.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-07T17:17:05.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-07T17:17:05.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-07T17:17:05.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-07T17:17:05.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-07T17:17:05.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-07T17:17:05.594 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-07T17:17:05.643 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.397735201Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2023-12-07T17:17:05.643 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.422877698Z level=info msg="Executing migration" id="Update json_data with nulls" 2023-12-07T17:17:05.643 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.464945794Z level=info msg="Executing migration" id="Add uid column" 2023-12-07T17:17:05.643 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.506804261Z level=info msg="Executing migration" id="Update uid value" 2023-12-07T17:17:05.643 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.532249335Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2023-12-07T17:17:05.644 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.557500853Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2023-12-07T17:17:05.644 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.599482798Z level=info msg="Executing migration" id="create api_key table" 2023-12-07T17:17:05.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:17:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:17:05.895 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.64147233Z level=info msg="Executing migration" id="add index api_key.account_id" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.683491647Z level=info msg="Executing migration" id="add index api_key.key" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.708803013Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.750911287Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.775984506Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.809632892Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.843173293Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.868412668Z level=info msg="Executing migration" id="create api_key table v2" 2023-12-07T17:17:05.896 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.893764787Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2023-12-07T17:17:06.165 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.944494756Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2023-12-07T17:17:06.165 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:05 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:05.969742526Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2023-12-07T17:17:06.165 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.011886559Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2023-12-07T17:17:06.165 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.037102813Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2023-12-07T17:17:06.166 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.0623091Z level=info msg="Executing migration" id="Update api_key table charset" 2023-12-07T17:17:06.166 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.087487941Z level=info msg="Executing migration" id="Add expires to api_key table" 2023-12-07T17:17:06.166 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.112777244Z level=info msg="Executing migration" id="Add service account foreign key" 2023-12-07T17:17:06.166 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.138045237Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2023-12-07T17:17:06.166 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.163281399Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2023-12-07T17:17:06.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.188605386Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2023-12-07T17:17:06.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.230636034Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2023-12-07T17:17:06.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.272685146Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2023-12-07T17:17:06.427 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.297721574Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2023-12-07T17:17:06.427 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.339950431Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2023-12-07T17:17:06.427 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.382115367Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2023-12-07T17:17:06.427 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.424274795Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.449262193Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.491534007Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.516713658Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.542082853Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.567263063Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.592549885Z level=info msg="Executing migration" id="create quota table v1" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.617726952Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2023-12-07T17:17:06.687 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.659937003Z level=info msg="Executing migration" id="Update quota table charset" 2023-12-07T17:17:06.948 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[31336]: cluster 2023-12-07T17:17:06.042181+0000 mgr.y (mgr.24626) 63 : cluster 0 pgmap v35: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:06.948 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[31336]: audit 2023-12-07T17:17:06.209984+0000 mon.c (mon.2) 54 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.685156459Z level=info msg="Executing migration" id="create plugin_setting table" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.72735135Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.769628863Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.79487793Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.836857373Z level=info msg="Executing migration" id="create session table" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.878959476Z level=info msg="Executing migration" id="Drop old table playlist table" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.921027604Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2023-12-07T17:17:06.949 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.946596126Z level=info msg="Executing migration" id="create playlist table v2" 2023-12-07T17:17:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:06 smithi064 bash[25056]: cluster 2023-12-07T17:17:06.042181+0000 mgr.y (mgr.24626) 63 : cluster 0 pgmap v35: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:06 smithi064 bash[25056]: audit 2023-12-07T17:17:06.209984+0000 mon.c (mon.2) 54 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:07.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:06 smithi064 bash[36207]: cluster 2023-12-07T17:17:06.042181+0000 mgr.y (mgr.24626) 63 : cluster 0 pgmap v35: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:17:07.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:06 smithi064 bash[36207]: audit 2023-12-07T17:17:06.209984+0000 mon.c (mon.2) 54 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:07.201 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:06 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:06.988383793Z level=info msg="Executing migration" id="create playlist item table v2" 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.013922532Z level=info msg="Executing migration" id="Update playlist table charset" 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.055848813Z level=info msg="Executing migration" id="Update playlist_item table charset" 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.081215542Z level=info msg="Executing migration" id="drop preferences table v2" 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.123287544Z level=info msg="Executing migration" id="drop preferences table v3" 2023-12-07T17:17:07.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.148629736Z level=info msg="Executing migration" id="create preferences table v3" 2023-12-07T17:17:07.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.173896956Z level=info msg="Executing migration" id="Update preferences table charset" 2023-12-07T17:17:07.461 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.199144712Z level=info msg="Executing migration" id="Add column team_id in preferences" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.257777166Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.283085832Z level=info msg="Executing migration" id="Add column week_start in preferences" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.30825218Z level=info msg="Executing migration" id="Add column preferences.json_data" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.333495753Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.375416939Z level=info msg="Executing migration" id="Add preferences index org_id" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.417542807Z level=info msg="Executing migration" id="Add preferences index user_id" 2023-12-07T17:17:07.462 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.459686698Z level=info msg="Executing migration" id="create alert table v1" 2023-12-07T17:17:07.722 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.48483696Z level=info msg="Executing migration" id="add index alert org_id & id " 2023-12-07T17:17:07.722 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.526830975Z level=info msg="Executing migration" id="add index alert state" 2023-12-07T17:17:07.722 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.552240531Z level=info msg="Executing migration" id="add index alert dashboard_id" 2023-12-07T17:17:07.722 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.577470188Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2023-12-07T17:17:07.723 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.611285187Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2023-12-07T17:17:07.723 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.636607365Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2023-12-07T17:17:07.723 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.678502928Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2023-12-07T17:17:07.723 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.720457009Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2023-12-07T17:17:07.974 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.034646+0000 mgr.y (mgr.24626) 64 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.349413+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.354237+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.366645+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.374477+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.376972+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.378407+0000 mon.c (mon.2) 56 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:07.975 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[31336]: audit 2023-12-07T17:17:07.387638+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:07.975 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.762611627Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2023-12-07T17:17:07.975 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.787774802Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2023-12-07T17:17:07.975 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.813037182Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2023-12-07T17:17:07.976 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.838249583Z level=info msg="Executing migration" id="create alert_notification table v1" 2023-12-07T17:17:07.976 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.863522619Z level=info msg="Executing migration" id="Add column is_default" 2023-12-07T17:17:07.976 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.905382079Z level=info msg="Executing migration" id="Add column frequency" 2023-12-07T17:17:07.976 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.930661335Z level=info msg="Executing migration" id="Add column send_reminder" 2023-12-07T17:17:07.976 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.972680339Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.034646+0000 mgr.y (mgr.24626) 64 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.349413+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.354237+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.366645+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.374477+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.376972+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.378407+0000 mon.c (mon.2) 56 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:08.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[36207]: audit 2023-12-07T17:17:07.387638+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.034646+0000 mgr.y (mgr.24626) 64 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.349413+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.354237+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.366645+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.374477+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.376972+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.378407+0000 mon.c (mon.2) 56 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:08.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:07 smithi064 bash[25056]: audit 2023-12-07T17:17:07.387638+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:08.235 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:07 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:07.99791493Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2023-12-07T17:17:08.235 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.023159829Z level=info msg="Executing migration" id="Update alert table charset" 2023-12-07T17:17:08.235 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.048455655Z level=info msg="Executing migration" id="Update alert_notification table charset" 2023-12-07T17:17:08.235 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.073696105Z level=info msg="Executing migration" id="create notification_journal table v1" 2023-12-07T17:17:08.235 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.098963221Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2023-12-07T17:17:08.236 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.124238924Z level=info msg="Executing migration" id="drop alert_notification_journal" 2023-12-07T17:17:08.236 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.166200931Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2023-12-07T17:17:08.236 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.191421749Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2023-12-07T17:17:08.236 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.233368483Z level=info msg="Executing migration" id="Add for to alert table" 2023-12-07T17:17:08.504 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.275404856Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2023-12-07T17:17:08.504 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.300663399Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2023-12-07T17:17:08.504 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.32595763Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2023-12-07T17:17:08.505 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.367894352Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2023-12-07T17:17:08.505 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.409958588Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2023-12-07T17:17:08.505 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.451890517Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2023-12-07T17:17:08.505 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.477105833Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2023-12-07T17:17:08.505 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.502411653Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2023-12-07T17:17:08.782 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[31336]: cluster 2023-12-07T17:17:07.381512+0000 mgr.y (mgr.24626) 65 : cluster 0 pgmap v36: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:08.782 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[31336]: cephadm 2023-12-07T17:17:07.465939+0000 mgr.y (mgr.24626) 66 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2023-12-07T17:17:08.782 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[31336]: cephadm 2023-12-07T17:17:07.473413+0000 mgr.y (mgr.24626) 67 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi064 2023-12-07T17:17:08.782 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.527673575Z level=info msg="Executing migration" id="Drop old annotation table v4" 2023-12-07T17:17:08.782 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.553303272Z level=info msg="Executing migration" id="create annotation table v5" 2023-12-07T17:17:08.782 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.578476602Z level=info msg="Executing migration" id="add index annotation 0 v3" 2023-12-07T17:17:08.782 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.603683677Z level=info msg="Executing migration" id="add index annotation 1 v3" 2023-12-07T17:17:08.782 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.628959543Z level=info msg="Executing migration" id="add index annotation 2 v3" 2023-12-07T17:17:08.783 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.670867082Z level=info msg="Executing migration" id="add index annotation 3 v3" 2023-12-07T17:17:08.783 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.696157737Z level=info msg="Executing migration" id="add index annotation 4 v3" 2023-12-07T17:17:08.783 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.738019247Z level=info msg="Executing migration" id="Update annotation table charset" 2023-12-07T17:17:08.783 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.780132822Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2023-12-07T17:17:08.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[25056]: cluster 2023-12-07T17:17:07.381512+0000 mgr.y (mgr.24626) 65 : cluster 0 pgmap v36: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:08.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[25056]: cephadm 2023-12-07T17:17:07.465939+0000 mgr.y (mgr.24626) 66 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2023-12-07T17:17:08.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[25056]: cephadm 2023-12-07T17:17:07.473413+0000 mgr.y (mgr.24626) 67 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi064 2023-12-07T17:17:08.995 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[36207]: cluster 2023-12-07T17:17:07.381512+0000 mgr.y (mgr.24626) 65 : cluster 0 pgmap v36: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:08.995 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[36207]: cephadm 2023-12-07T17:17:07.465939+0000 mgr.y (mgr.24626) 66 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2023-12-07T17:17:08.995 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:08 smithi064 bash[36207]: cephadm 2023-12-07T17:17:07.473413+0000 mgr.y (mgr.24626) 67 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi064 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.805335715Z level=info msg="Executing migration" id="Drop category_id index" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.830703063Z level=info msg="Executing migration" id="Add column tags to annotation table" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.872619275Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.898007652Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.964898389Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:08 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:08.990154301Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2023-12-07T17:17:09.051 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.023815419Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2023-12-07T17:17:09.317 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.049050259Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2023-12-07T17:17:09.317 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.074323988Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2023-12-07T17:17:09.317 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.099495979Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.141452671Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.174985506Z level=info msg="Executing migration" id="Add created time to annotation table" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.200325401Z level=info msg="Executing migration" id="Add updated time to annotation table" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.24222715Z level=info msg="Executing migration" id="Add index for created in annotation table" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.267567848Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2023-12-07T17:17:09.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.292823843Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.334782793Z level=info msg="Executing migration" id="Add epoch_end column" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.376902774Z level=info msg="Executing migration" id="Add index for epoch_end" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.418863918Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.460808706Z level=info msg="Executing migration" id="Move region to single row" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.502870978Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.536624916Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2023-12-07T17:17:09.589 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.562002518Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.587107199Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.612453177Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.63769527Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.679647499Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.704969616Z level=info msg="Executing migration" id="create test_data table" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.74676919Z level=info msg="Executing migration" id="create dashboard_version table v1" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.789025748Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.814371605Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2023-12-07T17:17:09.850 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.847922952Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.873074173Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.914991748Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.957080126Z level=info msg="Executing migration" id="create team table" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:09 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:09.982336812Z level=info msg="Executing migration" id="add index team.org_id" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.024366634Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.049537382Z level=info msg="Executing migration" id="create team member table" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.091612482Z level=info msg="Executing migration" id="add index team_member.org_id" 2023-12-07T17:17:10.119 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.116846284Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.15884439Z level=info msg="Executing migration" id="add index team_member.team_id" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.184060809Z level=info msg="Executing migration" id="Add column email to team table" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.209323885Z level=info msg="Executing migration" id="Add column external to team_member table" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.234651554Z level=info msg="Executing migration" id="Add column permission to team_member table" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.276528515Z level=info msg="Executing migration" id="create dashboard acl table" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.318580483Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.360598388Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2023-12-07T17:17:10.388 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.38592073Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2023-12-07T17:17:10.420 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.420 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:10.648 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.427820848Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.45343483Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.495225631Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.520609787Z level=info msg="Executing migration" id="add index dashboard_permission" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.545798071Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.571065758Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.5963371Z level=info msg="Executing migration" id="create tag table" 2023-12-07T17:17:10.649 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.621590172Z level=info msg="Executing migration" id="add index tag.key_value" 2023-12-07T17:17:10.934 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.646911816Z level=info msg="Executing migration" id="create login attempt table" 2023-12-07T17:17:10.934 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.688785154Z level=info msg="Executing migration" id="add index login_attempt.username" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.714015348Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.756025257Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.789816195Z level=info msg="Executing migration" id="create login_attempt v2" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.81493683Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.840209409Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.865414156Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.890689175Z level=info msg="Executing migration" id="create user auth table" 2023-12-07T17:17:10.935 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[31336]: cluster 2023-12-07T17:17:09.382243+0000 mgr.y (mgr.24626) 68 : cluster 0 pgmap v37: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:11.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:10 smithi064 bash[25056]: cluster 2023-12-07T17:17:09.382243+0000 mgr.y (mgr.24626) 68 : cluster 0 pgmap v37: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:11.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:10 smithi064 bash[36207]: cluster 2023-12-07T17:17:09.382243+0000 mgr.y (mgr.24626) 68 : cluster 0 pgmap v37: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:11.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.932564853Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.96626082Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:10 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:10.99146973Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.033466332Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.058735348Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.083924961Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.12601342Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.151193064Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2023-12-07T17:17:11.204 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.176426462Z level=info msg="Executing migration" id="create server_lock table" 2023-12-07T17:17:11.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.201675206Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2023-12-07T17:17:11.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.243785391Z level=info msg="Executing migration" id="create user auth token table" 2023-12-07T17:17:11.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.268980359Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2023-12-07T17:17:11.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.294061277Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2023-12-07T17:17:11.464 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.319212916Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2023-12-07T17:17:11.465 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.369969894Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2023-12-07T17:17:11.465 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.411980804Z level=info msg="Executing migration" id="create cache_data table" 2023-12-07T17:17:11.465 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.43710426Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2023-12-07T17:17:11.465 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.462271901Z level=info msg="Executing migration" id="create short_url table v1" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.487721104Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.529867521Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.571836497Z level=info msg="Executing migration" id="delete alert_definition table" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.597134689Z level=info msg="Executing migration" id="recreate alert_definition table" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.622352116Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.647682975Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.689652068Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2023-12-07T17:17:11.717 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.714908928Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.756834293Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.807125037Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.832500364Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.874437617Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.916442322Z level=info msg="Executing migration" id="drop alert_definition table" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.941741124Z level=info msg="Executing migration" id="delete alert_definition_version table" 2023-12-07T17:17:11.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:11 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:11.966962179Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2023-12-07T17:17:12.246 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.00900775Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.034446011Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.05966317Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.101563235Z level=info msg="Executing migration" id="drop alert_definition_version table" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.126877198Z level=info msg="Executing migration" id="create alert_instance table" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.152080408Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.177193883Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.202674127Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2023-12-07T17:17:12.247 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.244587114Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.270086988Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.311989636Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.362492126Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.396185323Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.421466001Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.446763172Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.472049214Z level=info msg="Executing migration" id="create alert_rule table" 2023-12-07T17:17:12.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.539024513Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[31336]: cluster 2023-12-07T17:17:11.383030+0000 mgr.y (mgr.24626) 69 : cluster 0 pgmap v38: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.564193431Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.589368032Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.614812609Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.640060008Z level=info msg="Executing migration" id="add column for to alert_rule" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.681928513Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2023-12-07T17:17:12.802 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.707094465Z level=info msg="Executing migration" id="add column labels to alert_rule" 2023-12-07T17:17:12.803 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.732430049Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2023-12-07T17:17:12.803 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.774604072Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2023-12-07T17:17:12.803 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.799788386Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2023-12-07T17:17:13.062 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.841781714Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.867100224Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.892358283Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.917738688Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.959679158Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:12 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:12.984711325Z level=info msg="Executing migration" id="create alert_rule_version table" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.009994049Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.035461949Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2023-12-07T17:17:13.063 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.060690127Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2023-12-07T17:17:13.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:12 smithi064 bash[25056]: cluster 2023-12-07T17:17:11.383030+0000 mgr.y (mgr.24626) 69 : cluster 0 pgmap v38: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:13.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:12 smithi064 bash[36207]: cluster 2023-12-07T17:17:11.383030+0000 mgr.y (mgr.24626) 69 : cluster 0 pgmap v38: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:13.315 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.085925612Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2023-12-07T17:17:13.315 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.127675625Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2023-12-07T17:17:13.315 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.152973648Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2023-12-07T17:17:13.315 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.178659174Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2023-12-07T17:17:13.315 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.204002612Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2023-12-07T17:17:13.316 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.245791694Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2023-12-07T17:17:13.316 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.28788036Z level=info msg="Executing migration" id=create_alert_configuration_table 2023-12-07T17:17:13.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.313220644Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.355084108Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.397168301Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.422581117Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.464589092Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.489756162Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2023-12-07T17:17:13.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.531756843Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.573572476Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.598931155Z level=info msg="Executing migration" id="create provenance_type table" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.624309211Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.64955961Z level=info msg="Executing migration" id="create alert_image table" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.674789795Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.708355307Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.741903375Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2023-12-07T17:17:13.828 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.783705925Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2023-12-07T17:17:13.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.825872982Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.826238977Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.851110681Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.8931548Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.935073832Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.935541516Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.935976775Z level=info msg="alerts found to migrate" alerts=0 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.960463054Z level=info msg="Executing migration" id="create library_element table v1" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:13 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:13.985759502Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.01098545Z level=info msg="Executing migration" id="create library_element_connection table v1" 2023-12-07T17:17:14.089 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.036176584Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2023-12-07T17:17:14.090 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.061454752Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2023-12-07T17:17:14.090 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.086688298Z level=info msg="Executing migration" id="increase max description length to 2048" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.120486187Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.162500658Z level=info msg="Executing migration" id="create data_keys table" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.187690411Z level=info msg="Executing migration" id="create secrets table" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.229751437Z level=info msg="Executing migration" id="rename data_keys name column to id" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.263267746Z level=info msg="Executing migration" id="add name column into data_keys" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.28857824Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2023-12-07T17:17:14.349 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.347134445Z level=info msg="Executing migration" id="rename data_keys name column to label" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.397545966Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.431012515Z level=info msg="Executing migration" id="create kv_store table v1" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.456516199Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.481706829Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.506752417Z level=info msg="Executing migration" id="create permission table" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.548734004Z level=info msg="Executing migration" id="add unique index permission.role_id" 2023-12-07T17:17:14.618 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.59060687Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2023-12-07T17:17:14.619 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.615903787Z level=info msg="Executing migration" id="create role table" 2023-12-07T17:17:14.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[31336]: cluster 2023-12-07T17:17:13.384067+0000 mgr.y (mgr.24626) 70 : cluster 0 pgmap v39: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:14.887 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.641286541Z level=info msg="Executing migration" id="add column display_name" 2023-12-07T17:17:14.887 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.68326384Z level=info msg="Executing migration" id="add column group_name" 2023-12-07T17:17:14.887 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.708427238Z level=info msg="Executing migration" id="add index role.org_id" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.733735248Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.775971363Z level=info msg="Executing migration" id="add index role_org_id_uid" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.792924284Z level=info msg="Executing migration" id="create team role table" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.818232996Z level=info msg="Executing migration" id="add index team_role.org_id" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.843354295Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2023-12-07T17:17:14.888 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.885330734Z level=info msg="Executing migration" id="add index team_role.team_id" 2023-12-07T17:17:15.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:14 smithi064 bash[25056]: cluster 2023-12-07T17:17:13.384067+0000 mgr.y (mgr.24626) 70 : cluster 0 pgmap v39: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:15.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:14 smithi064 bash[36207]: cluster 2023-12-07T17:17:13.384067+0000 mgr.y (mgr.24626) 70 : cluster 0 pgmap v39: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:15.148 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.927382998Z level=info msg="Executing migration" id="create user role table" 2023-12-07T17:17:15.148 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.952618697Z level=info msg="Executing migration" id="add index user_role.org_id" 2023-12-07T17:17:15.148 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:14 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:14.977875013Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2023-12-07T17:17:15.148 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.002805383Z level=info msg="Executing migration" id="add index user_role.user_id" 2023-12-07T17:17:15.148 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.028188256Z level=info msg="Executing migration" id="create builtin role table" 2023-12-07T17:17:15.149 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.053547046Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2023-12-07T17:17:15.149 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.0789823Z level=info msg="Executing migration" id="add index builtin_role.name" 2023-12-07T17:17:15.149 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.120927822Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2023-12-07T17:17:15.149 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.146245617Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.188170897Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.230145117Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.263721429Z level=info msg="Executing migration" id="add unique index role.uid" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.288907276Z level=info msg="Executing migration" id="create seed assignment table" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.314286488Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.35643895Z level=info msg="Executing migration" id="add column hidden to role table" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.381647733Z level=info msg="Executing migration" id="create query_history table v1" 2023-12-07T17:17:15.426 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.423686596Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.448856416Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.47412149Z level=info msg="Executing migration" id="teams permissions migration" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.499534266Z level=info msg="Executing migration" id="dashboard permissions" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.541394632Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.583384546Z level=info msg="Executing migration" id="drop managed folder create actions" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.608617278Z level=info msg="Executing migration" id="alerting notification permissions" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.633889786Z level=info msg="Executing migration" id="create query_history_star table v1" 2023-12-07T17:17:15.678 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.675924252Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2023-12-07T17:17:15.689 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:15 smithi064 systemd[1]: Stopping Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:17:15.869 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:17:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.701191694Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.726340707Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.768391393Z level=info msg="Executing migration" id="create correlation table v1" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.793727013Z level=info msg="Executing migration" id="add index correlations.uid" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.835621963Z level=info msg="Executing migration" id="add index correlations.source_uid" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.877543447Z level=info msg="Executing migration" id="add correlation config column" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.919716885Z level=info msg="Executing migration" id="create entity_events table" 2023-12-07T17:17:15.947 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.944950731Z level=info msg="Executing migration" id="create dashboard public config v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.970232298Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.970571997Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.995661807Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:15 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:15.996002968Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.020952538Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.046281984Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.113049135Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2023-12-07T17:17:16.233 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.138537911Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2023-12-07T17:17:16.234 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.163704621Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2023-12-07T17:17:16.234 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.188868978Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2023-12-07T17:17:16.234 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.230861288Z level=info msg="Executing migration" id="Drop public config table" 2023-12-07T17:17:16.493 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.256186368Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2023-12-07T17:17:16.493 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.2814141Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2023-12-07T17:17:16.493 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.306673972Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2023-12-07T17:17:16.494 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.348582241Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2023-12-07T17:17:16.494 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.39057376Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2023-12-07T17:17:16.494 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.440911892Z level=info msg="Executing migration" id="add annotations_enabled column" 2023-12-07T17:17:16.494 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.466224532Z level=info msg="Executing migration" id="add time_selection_enabled column" 2023-12-07T17:17:16.494 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.491359902Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2023-12-07T17:17:16.757 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.516664797Z level=info msg="Executing migration" id="add share column" 2023-12-07T17:17:16.757 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.541959314Z level=info msg="Executing migration" id="create default alerting folders" 2023-12-07T17:17:16.757 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.583891302Z level=info msg="Executing migration" id="create file table" 2023-12-07T17:17:16.758 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.609261731Z level=info msg="Executing migration" id="file table idx: path natural pk" 2023-12-07T17:17:16.758 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.634502349Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2023-12-07T17:17:16.758 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.676497436Z level=info msg="Executing migration" id="create file_meta table" 2023-12-07T17:17:16.758 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.701683859Z level=info msg="Executing migration" id="file table idx: path key" 2023-12-07T17:17:16.758 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.726949402Z level=info msg="Executing migration" id="set path collation in file table" 2023-12-07T17:17:17.015 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[31336]: cluster 2023-12-07T17:17:15.384546+0000 mgr.y (mgr.24626) 71 : cluster 0 pgmap v40: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:17.015 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.768880021Z level=info msg="Executing migration" id="managed permissions migration" 2023-12-07T17:17:17.015 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.794288249Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2023-12-07T17:17:17.015 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.836212271Z level=info msg="Executing migration" id="RBAC action name migrator" 2023-12-07T17:17:17.015 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.861513558Z level=info msg="Executing migration" id="Add UID column to playlist" 2023-12-07T17:17:17.016 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.886790776Z level=info msg="Executing migration" id="Update uid column values in playlist" 2023-12-07T17:17:17.016 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.911912469Z level=info msg="Executing migration" id="Add index for uid in playlist" 2023-12-07T17:17:17.016 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.945880642Z level=info msg="Executing migration" id="update group index for alert rules" 2023-12-07T17:17:17.016 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:16 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:16.970939364Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2023-12-07T17:17:17.016 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.013035614Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2023-12-07T17:17:17.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:16 smithi064 bash[25056]: cluster 2023-12-07T17:17:15.384546+0000 mgr.y (mgr.24626) 71 : cluster 0 pgmap v40: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:17.259 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:16 smithi064 bash[36207]: cluster 2023-12-07T17:17:15.384546+0000 mgr.y (mgr.24626) 71 : cluster 0 pgmap v40: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2023-12-07T17:17:17.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.038256339Z level=info msg="Executing migration" id="add action column to seed_assignment" 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.080187345Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.105464905Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.130632709Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.172991862Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2023-12-07T17:17:17.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.199018902Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2023-12-07T17:17:17.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.241068902Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2023-12-07T17:17:17.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.276269578Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2023-12-07T17:17:17.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.301409485Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2023-12-07T17:17:17.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.326722301Z level=info msg="Executing migration" id="create folder table" 2023-12-07T17:17:17.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.369042827Z level=info msg="Executing migration" id="Add index for parent_uid" 2023-12-07T17:17:17.567 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.41095876Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2023-12-07T17:17:17.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.436326506Z level=info msg="Executing migration" id="Update folder title length" 2023-12-07T17:17:17.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.478251194Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2023-12-07T17:17:17.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=migrator t=2023-12-07T17:17:17.503501032Z level=info msg="migrations completed" performed=485 skipped=0 duration=15.620440212s 2023-12-07T17:17:17.568 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=sqlstore t=2023-12-07T17:17:17.504896428Z level=info msg="Created default organization" 2023-12-07T17:17:17.598 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:17 smithi064 bash[93704]: ts=2023-12-07T17:17:17.274Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2023-12-07T17:17:17.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:17 smithi064 bash[25056]: audit 2023-12-07T17:17:17.042878+0000 mgr.y (mgr.24626) 72 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:17.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:17 smithi064 bash[36207]: audit 2023-12-07T17:17:17.042878+0000 mgr.y (mgr.24626) 72 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:18.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[31336]: audit 2023-12-07T17:17:17.042878+0000 mgr.y (mgr.24626) 72 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=plugin.loader t=2023-12-07T17:17:17.629440388Z level=info msg="Plugin registered" pluginID=input 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=plugin.loader t=2023-12-07T17:17:17.658205658Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=plugin.loader t=2023-12-07T17:17:17.658226437Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=secrets t=2023-12-07T17:17:17.658264151Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=query_data t=2023-12-07T17:17:17.660162033Z level=info msg="Query Service initialization" 2023-12-07T17:17:18.068 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:17 smithi184 bash[82556]: logger=live.push_http t=2023-12-07T17:17:17.663982792Z level=info msg="Live Push Gateway initialization" 2023-12-07T17:17:18.901 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[31336]: cluster 2023-12-07T17:17:17.385048+0000 mgr.y (mgr.24626) 73 : cluster 0 pgmap v41: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=infra.usagestats.collector t=2023-12-07T17:17:18.605560111Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=provisioning.datasources t=2023-12-07T17:17:18.66269442Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=provisioning.datasources t=2023-12-07T17:17:18.732001484Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=provisioning.alerting t=2023-12-07T17:17:18.806499021Z level=info msg="starting to provision alerting" 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=provisioning.alerting t=2023-12-07T17:17:18.806568263Z level=info msg="finished to provision alerting" 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=grafanaStorageLogger t=2023-12-07T17:17:18.806956373Z level=info msg="storage starting" 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=ngalert.state.manager t=2023-12-07T17:17:18.8070583Z level=info msg="Warming state cache for startup" 2023-12-07T17:17:18.902 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=http.server t=2023-12-07T17:17:18.809398361Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2023-12-07T17:17:18.903 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=ngalert.state.manager t=2023-12-07T17:17:18.899006908Z level=info msg="State cache has been initialized" states=0 duration=91.945235ms 2023-12-07T17:17:18.903 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=ngalert.multiorg.alertmanager t=2023-12-07T17:17:18.899117916Z level=info msg="starting MultiOrg Alertmanager" 2023-12-07T17:17:19.074 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:18 smithi064 bash[36207]: cluster 2023-12-07T17:17:17.385048+0000 mgr.y (mgr.24626) 73 : cluster 0 pgmap v41: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:19.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:18 smithi064 bash[25056]: cluster 2023-12-07T17:17:17.385048+0000 mgr.y (mgr.24626) 73 : cluster 0 pgmap v41: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2023-12-07T17:17:19.317 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=ticker t=2023-12-07T17:17:18.899177409Z level=info msg=starting first_tick=2023-12-07T17:17:20Z 2023-12-07T17:17:19.317 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:17:18 smithi184 bash[82556]: logger=sqlstore.transactions t=2023-12-07T17:17:18.969909057Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2023-12-07T17:17:21.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:20 smithi184 bash[31336]: cluster 2023-12-07T17:17:19.385592+0000 mgr.y (mgr.24626) 74 : cluster 0 pgmap v42: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:21.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:20 smithi064 bash[25056]: cluster 2023-12-07T17:17:19.385592+0000 mgr.y (mgr.24626) 74 : cluster 0 pgmap v42: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:21.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:20 smithi064 bash[36207]: cluster 2023-12-07T17:17:19.385592+0000 mgr.y (mgr.24626) 74 : cluster 0 pgmap v42: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:22.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:21 smithi184 bash[31336]: audit 2023-12-07T17:17:21.227579+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:22.149 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:21 smithi064 bash[36207]: audit 2023-12-07T17:17:21.227579+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:22.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:21 smithi064 bash[25056]: audit 2023-12-07T17:17:21.227579+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:23.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:22 smithi184 bash[31336]: cluster 2023-12-07T17:17:21.386159+0000 mgr.y (mgr.24626) 75 : cluster 0 pgmap v43: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:23.195 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:22 smithi064 bash[25056]: cluster 2023-12-07T17:17:21.386159+0000 mgr.y (mgr.24626) 75 : cluster 0 pgmap v43: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:23.195 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:22 smithi064 bash[36207]: cluster 2023-12-07T17:17:21.386159+0000 mgr.y (mgr.24626) 75 : cluster 0 pgmap v43: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:23.195 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:22 smithi064 bash[96036]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-alertmanager-a 2023-12-07T17:17:25.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:24 smithi184 bash[31336]: cluster 2023-12-07T17:17:23.386882+0000 mgr.y (mgr.24626) 76 : cluster 0 pgmap v44: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:25.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:24 smithi064 bash[25056]: cluster 2023-12-07T17:17:23.386882+0000 mgr.y (mgr.24626) 76 : cluster 0 pgmap v44: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:25.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:24 smithi064 bash[36207]: cluster 2023-12-07T17:17:23.386882+0000 mgr.y (mgr.24626) 76 : cluster 0 pgmap v44: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:26.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:17:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:17:26.888 INFO:teuthology.orchestra.run.smithi064.stdout:163208757307 2023-12-07T17:17:26.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-07T17:17:27.049 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:26 smithi184 bash[31336]: cluster 2023-12-07T17:17:25.387316+0000 mgr.y (mgr.24626) 77 : cluster 0 pgmap v45: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:27.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:26 smithi064 bash[25056]: cluster 2023-12-07T17:17:25.387316+0000 mgr.y (mgr.24626) 77 : cluster 0 pgmap v45: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:27.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:26 smithi064 bash[36207]: cluster 2023-12-07T17:17:25.387316+0000 mgr.y (mgr.24626) 77 : cluster 0 pgmap v45: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:27.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:28.036 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[25056]: audit 2023-12-07T17:17:27.046735+0000 mgr.y (mgr.24626) 78 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:28.036 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[25056]: audit 2023-12-07T17:17:27.771425+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:28.036 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[25056]: audit 2023-12-07T17:17:27.780397+0000 mon.a (mon.0) 800 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:28.036 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[36207]: audit 2023-12-07T17:17:27.046735+0000 mgr.y (mgr.24626) 78 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:28.036 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[36207]: audit 2023-12-07T17:17:27.771425+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:28.037 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:27 smithi064 bash[36207]: audit 2023-12-07T17:17:27.780397+0000 mon.a (mon.0) 800 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:28.037 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:27 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@alertmanager.a.service: Succeeded. 2023-12-07T17:17:28.037 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:27 smithi064 systemd[1]: Stopped Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:17:28.037 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:27 smithi064 systemd[1]: Started Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:17:28.038 INFO:teuthology.orchestra.run.smithi064.stdout:34359738482 2023-12-07T17:17:28.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-07T17:17:28.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:27 smithi184 bash[31336]: audit 2023-12-07T17:17:27.046735+0000 mgr.y (mgr.24626) 78 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:28.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:27 smithi184 bash[31336]: audit 2023-12-07T17:17:27.771425+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:28.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:27 smithi184 bash[31336]: audit 2023-12-07T17:17:27.780397+0000 mon.a (mon.0) 800 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:29.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:28 smithi184 bash[31336]: cluster 2023-12-07T17:17:27.388301+0000 mgr.y (mgr.24626) 79 : cluster 0 pgmap v46: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:28 smithi184 bash[31336]: cephadm 2023-12-07T17:17:27.834936+0000 mgr.y (mgr.24626) 80 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2023-12-07T17:17:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:28 smithi184 bash[31336]: cephadm 2023-12-07T17:17:28.097867+0000 mgr.y (mgr.24626) 81 : cephadm 1 Reconfiguring daemon prometheus.a on smithi184 2023-12-07T17:17:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[25056]: cluster 2023-12-07T17:17:27.388301+0000 mgr.y (mgr.24626) 79 : cluster 0 pgmap v46: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[25056]: cephadm 2023-12-07T17:17:27.834936+0000 mgr.y (mgr.24626) 80 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2023-12-07T17:17:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[25056]: cephadm 2023-12-07T17:17:28.097867+0000 mgr.y (mgr.24626) 81 : cephadm 1 Reconfiguring daemon prometheus.a on smithi184 2023-12-07T17:17:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[36207]: cluster 2023-12-07T17:17:27.388301+0000 mgr.y (mgr.24626) 79 : cluster 0 pgmap v46: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[36207]: cephadm 2023-12-07T17:17:27.834936+0000 mgr.y (mgr.24626) 80 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2023-12-07T17:17:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:28 smithi064 bash[36207]: cephadm 2023-12-07T17:17:28.097867+0000 mgr.y (mgr.24626) 81 : cephadm 1 Reconfiguring daemon prometheus.a on smithi184 2023-12-07T17:17:30.606 INFO:teuthology.orchestra.run.smithi064.stdout:231928234019 2023-12-07T17:17:30.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-07T17:17:30.796 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:30 smithi064 bash[36207]: cluster 2023-12-07T17:17:29.389055+0000 mgr.y (mgr.24626) 82 : cluster 0 pgmap v47: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:30.915 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:30 smithi064 bash[25056]: cluster 2023-12-07T17:17:29.389055+0000 mgr.y (mgr.24626) 82 : cluster 0 pgmap v47: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:31.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:30 smithi184 bash[31336]: cluster 2023-12-07T17:17:29.389055+0000 mgr.y (mgr.24626) 82 : cluster 0 pgmap v47: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:32.012 INFO:teuthology.orchestra.run.smithi064.stdout:107374182480 2023-12-07T17:17:32.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-07T17:17:32.317 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:31 smithi184 systemd[1]: Stopping Ceph prometheus.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:17:32.317 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-12-07T17:17:32.317 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2023-12-07T17:17:32.317 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.133Z caller=main.go:907 level=info msg="Scrape manager stopped" 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.134Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.134Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2023-12-07T17:17:32.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[80576]: ts=2023-12-07T17:17:32.134Z caller=main.go:1147 level=info msg="See you next time!" 2023-12-07T17:17:32.909 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[31336]: cluster 2023-12-07T17:17:31.389561+0000 mgr.y (mgr.24626) 83 : cluster 0 pgmap v48: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:32.909 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[83206]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-prometheus-a 2023-12-07T17:17:32.909 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 bash[83270]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-prometheus-a 2023-12-07T17:17:32.909 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@prometheus.a.service: Succeeded. 2023-12-07T17:17:32.909 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 systemd[1]: Stopped Ceph prometheus.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:17:32.909 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:32 smithi184 systemd[1]: Started Ceph prometheus.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:17:33.037 INFO:teuthology.orchestra.run.smithi064.stdout:197568495664 2023-12-07T17:17:33.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-07T17:17:33.069 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:32 smithi064 bash[25333]: [07/Dec/2023:17:17:32] ENGINE Bus STOPPING 2023-12-07T17:17:33.069 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-12-07T17:17:33.069 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:32] ENGINE Bus STOPPED 2023-12-07T17:17:33.069 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:32] ENGINE Bus STARTING 2023-12-07T17:17:33.069 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:32 smithi064 bash[36207]: cluster 2023-12-07T17:17:31.389561+0000 mgr.y (mgr.24626) 83 : cluster 0 pgmap v48: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:33.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:32 smithi064 bash[25056]: cluster 2023-12-07T17:17:31.389561+0000 mgr.y (mgr.24626) 83 : cluster 0 pgmap v48: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2023-12-07T17:17:33.369 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Serving on http://:::9283 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STARTED 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STOPPING 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STOPPED 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STARTING 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Serving on http://:::9283 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STARTED 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STOPPING 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STOPPED 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STARTING 2023-12-07T17:17:33.370 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Serving on http://:::9283 2023-12-07T17:17:33.371 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25333]: [07/Dec/2023:17:17:33] ENGINE Bus STARTED 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.894512+0000 mon.a (mon.0) 801 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.905712+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.930193+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.930769+0000 mgr.y (mgr.24626) 84 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.932263+0000 mon.c (mon.2) 59 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.932686+0000 mgr.y (mgr.24626) 85 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.941217+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.956939+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.957494+0000 mgr.y (mgr.24626) 86 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.960901+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.961402+0000 mgr.y (mgr.24626) 87 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.969768+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.980537+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.981141+0000 mgr.y (mgr.24626) 88 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.985095+0000 mon.c (mon.2) 63 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.985536+0000 mgr.y (mgr.24626) 89 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:32.992336+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.069 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[31336]: audit 2023-12-07T17:17:33.013595+0000 mon.c (mon.2) 64 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:34.069 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.872Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.872Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.872Z caller=main.go:570 level=info host_details="(Linux 5.15.0-89-generic #99~20.04.1-Ubuntu SMP Thu Nov 2 15:16:47 UTC 2023 x86_64 smithi184 (none))" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.872Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.872Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.874Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.875Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.876Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.876Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.881Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.881Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.381µs 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.881Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.882Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.882Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2023-12-07T17:17:34.070 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.882Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.483µs wal_replay_duration=930.217µs wbl_replay_duration=130ns total_replay_duration=979.346µs 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.883Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.883Z caller=main.go:1029 level=info msg="TSDB started" 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.883Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.913Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=30.229522ms db_storage=1.104µs remote_storage=1.564µs web_handler=590ns query_engine=831ns scrape=374.089µs scrape_sd=124.089µs notify=16.555µs notify_sd=15.231µs rules=29.329661ms tracing=9.127µs 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.913Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2023-12-07T17:17:34.071 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:17:33 smithi184 bash[83299]: ts=2023-12-07T17:17:33.913Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2023-12-07T17:17:34.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.894512+0000 mon.a (mon.0) 801 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.905712+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.930193+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.930769+0000 mgr.y (mgr.24626) 84 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.932263+0000 mon.c (mon.2) 59 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.932686+0000 mgr.y (mgr.24626) 85 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.941217+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.956939+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.957494+0000 mgr.y (mgr.24626) 86 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.960901+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.961402+0000 mgr.y (mgr.24626) 87 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.969768+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.980537+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.981141+0000 mgr.y (mgr.24626) 88 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.985095+0000 mon.c (mon.2) 63 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.985536+0000 mgr.y (mgr.24626) 89 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:32.992336+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[36207]: audit 2023-12-07T17:17:33.013595+0000 mon.c (mon.2) 64 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.894512+0000 mon.a (mon.0) 801 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.905712+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.930193+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.930769+0000 mgr.y (mgr.24626) 84 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.932263+0000 mon.c (mon.2) 59 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.932686+0000 mgr.y (mgr.24626) 85 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi064.front.sepia.ceph.com:9093"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.941217+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.956939+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.957494+0000 mgr.y (mgr.24626) 86 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.960901+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.961402+0000 mgr.y (mgr.24626) 87 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.969768+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.980537+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.981141+0000 mgr.y (mgr.24626) 88 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.985095+0000 mon.c (mon.2) 63 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.985536+0000 mgr.y (mgr.24626) 89 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:32.992336+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:34.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:33 smithi064 bash[25056]: audit 2023-12-07T17:17:33.013595+0000 mon.c (mon.2) 64 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:17:34.132 INFO:teuthology.orchestra.run.smithi064.stdout:55834574951 2023-12-07T17:17:34.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-07T17:17:34.340 INFO:teuthology.orchestra.run.smithi064.stdout:128849018950 2023-12-07T17:17:34.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-07T17:17:34.676 INFO:teuthology.orchestra.run.smithi064.stdout:77309411420 2023-12-07T17:17:34.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.683Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.683Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.685Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.64 port=9094 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.686Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.725Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.725Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.730Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2023-12-07T17:17:34.870 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[96727]: ts=2023-12-07T17:17:34.730Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2023-12-07T17:17:35.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:34 smithi184 bash[31336]: cluster 2023-12-07T17:17:33.390131+0000 mgr.y (mgr.24626) 90 : cluster 0 pgmap v49: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:35.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[25056]: cluster 2023-12-07T17:17:33.390131+0000 mgr.y (mgr.24626) 90 : cluster 0 pgmap v49: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:35.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:34 smithi064 bash[36207]: cluster 2023-12-07T17:17:33.390131+0000 mgr.y (mgr.24626) 90 : cluster 0 pgmap v49: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:37.005 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:36 smithi064 bash[96727]: ts=2023-12-07T17:17:36.686Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000125284s 2023-12-07T17:17:37.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:37.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:37 smithi184 bash[31336]: cluster 2023-12-07T17:17:35.390748+0000 mgr.y (mgr.24626) 91 : cluster 0 pgmap v50: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:37.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:37 smithi184 bash[31336]: audit 2023-12-07T17:17:36.245696+0000 mon.c (mon.2) 65 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:37.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:37 smithi064 bash[25056]: cluster 2023-12-07T17:17:35.390748+0000 mgr.y (mgr.24626) 91 : cluster 0 pgmap v50: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:37.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:37 smithi064 bash[25056]: audit 2023-12-07T17:17:36.245696+0000 mon.c (mon.2) 65 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:37.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:37 smithi064 bash[36207]: cluster 2023-12-07T17:17:35.390748+0000 mgr.y (mgr.24626) 91 : cluster 0 pgmap v50: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:37.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:37 smithi064 bash[36207]: audit 2023-12-07T17:17:36.245696+0000 mon.c (mon.2) 65 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:37.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:37.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:37.717 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:37.718 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:38.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:38 smithi184 bash[31336]: audit 2023-12-07T17:17:37.056054+0000 mgr.y (mgr.24626) 92 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:38.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:38 smithi064 bash[25056]: audit 2023-12-07T17:17:37.056054+0000 mgr.y (mgr.24626) 92 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:38.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:38 smithi064 bash[36207]: audit 2023-12-07T17:17:37.056054+0000 mgr.y (mgr.24626) 92 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:38.729 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:38.730 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:38.731 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:38.733 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:17:39.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:39 smithi184 bash[31336]: cluster 2023-12-07T17:17:37.391645+0000 mgr.y (mgr.24626) 93 : cluster 0 pgmap v51: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:39.369 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:39 smithi064 bash[36207]: cluster 2023-12-07T17:17:37.391645+0000 mgr.y (mgr.24626) 93 : cluster 0 pgmap v51: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:39.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:39 smithi064 bash[25056]: cluster 2023-12-07T17:17:37.391645+0000 mgr.y (mgr.24626) 93 : cluster 0 pgmap v51: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: cluster 2023-12-07T17:17:39.392242+0000 mgr.y (mgr.24626) 94 : cluster 0 pgmap v52: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:39.603127+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:39.612604+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:39.654375+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:39.667638+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:40.243926+0000 mon.c (mon.2) 66 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:40.244780+0000 mon.c (mon.2) 67 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:40.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[36207]: audit 2023-12-07T17:17:40.252695+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: cluster 2023-12-07T17:17:39.392242+0000 mgr.y (mgr.24626) 94 : cluster 0 pgmap v52: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:39.603127+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:39.612604+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:39.654375+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:39.667638+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:40.243926+0000 mon.c (mon.2) 66 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:40.244780+0000 mon.c (mon.2) 67 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:40.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:40 smithi064 bash[25056]: audit 2023-12-07T17:17:40.252695+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:41.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: cluster 2023-12-07T17:17:39.392242+0000 mgr.y (mgr.24626) 94 : cluster 0 pgmap v52: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:41.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:39.603127+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:39.612604+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:39.654375+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:39.667638+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:40.243926+0000 mon.c (mon.2) 66 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:40.244780+0000 mon.c (mon.2) 67 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:17:41.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:40 smithi184 bash[31336]: audit 2023-12-07T17:17:40.252695+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:17:42.182 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:42 smithi064 bash[25056]: cluster 2023-12-07T17:17:40.246261+0000 mgr.y (mgr.24626) 95 : cluster 0 pgmap v53: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 848 B/s rd, 0 op/s 2023-12-07T17:17:42.182 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:42 smithi064 bash[25056]: cluster 2023-12-07T17:17:40.246447+0000 mgr.y (mgr.24626) 96 : cluster 0 pgmap v54: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-12-07T17:17:42.182 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:42 smithi064 bash[36207]: cluster 2023-12-07T17:17:40.246261+0000 mgr.y (mgr.24626) 95 : cluster 0 pgmap v53: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 848 B/s rd, 0 op/s 2023-12-07T17:17:42.183 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:42 smithi064 bash[36207]: cluster 2023-12-07T17:17:40.246447+0000 mgr.y (mgr.24626) 96 : cluster 0 pgmap v54: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-12-07T17:17:42.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:42 smithi184 bash[31336]: cluster 2023-12-07T17:17:40.246261+0000 mgr.y (mgr.24626) 95 : cluster 0 pgmap v53: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 848 B/s rd, 0 op/s 2023-12-07T17:17:42.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:42 smithi184 bash[31336]: cluster 2023-12-07T17:17:40.246447+0000 mgr.y (mgr.24626) 96 : cluster 0 pgmap v54: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2023-12-07T17:17:44.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:44 smithi064 bash[36207]: cluster 2023-12-07T17:17:42.247190+0000 mgr.y (mgr.24626) 97 : cluster 0 pgmap v55: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:44.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:44 smithi064 bash[25056]: cluster 2023-12-07T17:17:42.247190+0000 mgr.y (mgr.24626) 97 : cluster 0 pgmap v55: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:44.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:44 smithi184 bash[31336]: cluster 2023-12-07T17:17:42.247190+0000 mgr.y (mgr.24626) 97 : cluster 0 pgmap v55: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:44.969 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:17:44 smithi064 bash[96727]: ts=2023-12-07T17:17:44.688Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001732645s 2023-12-07T17:17:46.028 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:17:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:17:46.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:46 smithi184 bash[31336]: cluster 2023-12-07T17:17:44.247589+0000 mgr.y (mgr.24626) 98 : cluster 0 pgmap v56: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:46.369 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:46 smithi064 bash[25056]: cluster 2023-12-07T17:17:44.247589+0000 mgr.y (mgr.24626) 98 : cluster 0 pgmap v56: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:46.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:46 smithi064 bash[36207]: cluster 2023-12-07T17:17:44.247589+0000 mgr.y (mgr.24626) 98 : cluster 0 pgmap v56: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:47.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:48.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:48 smithi184 bash[31336]: cluster 2023-12-07T17:17:46.248215+0000 mgr.y (mgr.24626) 99 : cluster 0 pgmap v57: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2023-12-07T17:17:48.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:48 smithi184 bash[31336]: audit 2023-12-07T17:17:47.067430+0000 mgr.y (mgr.24626) 100 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:48.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:48 smithi064 bash[36207]: cluster 2023-12-07T17:17:46.248215+0000 mgr.y (mgr.24626) 99 : cluster 0 pgmap v57: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2023-12-07T17:17:48.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:48 smithi064 bash[36207]: audit 2023-12-07T17:17:47.067430+0000 mgr.y (mgr.24626) 100 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:48.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:48 smithi064 bash[25056]: cluster 2023-12-07T17:17:46.248215+0000 mgr.y (mgr.24626) 99 : cluster 0 pgmap v57: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 0 op/s 2023-12-07T17:17:48.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:48 smithi064 bash[25056]: audit 2023-12-07T17:17:47.067430+0000 mgr.y (mgr.24626) 100 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:49.947 INFO:teuthology.orchestra.run.smithi064.stdout:107374182484 2023-12-07T17:17:50.299 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:50 smithi064 bash[25056]: cluster 2023-12-07T17:17:48.248840+0000 mgr.y (mgr.24626) 101 : cluster 0 pgmap v58: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:50.299 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:50 smithi064 bash[25056]: audit 2023-12-07T17:17:49.945262+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.64:0/3733671434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T17:17:50.299 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:50 smithi064 bash[36207]: cluster 2023-12-07T17:17:48.248840+0000 mgr.y (mgr.24626) 101 : cluster 0 pgmap v58: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:50.299 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:50 smithi064 bash[36207]: audit 2023-12-07T17:17:49.945262+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.64:0/3733671434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T17:17:50.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:50 smithi184 bash[31336]: cluster 2023-12-07T17:17:48.248840+0000 mgr.y (mgr.24626) 101 : cluster 0 pgmap v58: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:17:50.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:50 smithi184 bash[31336]: audit 2023-12-07T17:17:49.945262+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.64:0/3733671434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T17:17:51.694 INFO:teuthology.orchestra.run.smithi064.stdout:197568495668 2023-12-07T17:17:51.999 INFO:teuthology.orchestra.run.smithi064.stdout:163208757312 2023-12-07T17:17:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:52 smithi184 bash[31336]: cluster 2023-12-07T17:17:50.249457+0000 mgr.y (mgr.24626) 102 : cluster 0 pgmap v59: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:17:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:52 smithi184 bash[31336]: audit 2023-12-07T17:17:51.263986+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:52 smithi184 bash[31336]: audit 2023-12-07T17:17:51.691729+0000 mon.c (mon.2) 70 : audit 0 from='client.? 172.21.15.64:0/2943640966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T17:17:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:52 smithi184 bash[31336]: audit 2023-12-07T17:17:51.997595+0000 mon.a (mon.0) 811 : audit 0 from='client.? 172.21.15.64:0/491165093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T17:17:52.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[36207]: cluster 2023-12-07T17:17:50.249457+0000 mgr.y (mgr.24626) 102 : cluster 0 pgmap v59: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:17:52.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[36207]: audit 2023-12-07T17:17:51.263986+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:52.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[36207]: audit 2023-12-07T17:17:51.691729+0000 mon.c (mon.2) 70 : audit 0 from='client.? 172.21.15.64:0/2943640966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T17:17:52.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[36207]: audit 2023-12-07T17:17:51.997595+0000 mon.a (mon.0) 811 : audit 0 from='client.? 172.21.15.64:0/491165093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T17:17:52.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[25056]: cluster 2023-12-07T17:17:50.249457+0000 mgr.y (mgr.24626) 102 : cluster 0 pgmap v59: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:17:52.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[25056]: audit 2023-12-07T17:17:51.263986+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:17:52.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[25056]: audit 2023-12-07T17:17:51.691729+0000 mon.c (mon.2) 70 : audit 0 from='client.? 172.21.15.64:0/2943640966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T17:17:52.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:52 smithi064 bash[25056]: audit 2023-12-07T17:17:51.997595+0000 mon.a (mon.0) 811 : audit 0 from='client.? 172.21.15.64:0/491165093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T17:17:53.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182480 got 107374182484 for osd.3 2023-12-07T17:17:53.403 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:54.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:54 smithi184 bash[31336]: cluster 2023-12-07T17:17:52.250054+0000 mgr.y (mgr.24626) 103 : cluster 0 pgmap v60: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:54.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:54 smithi064 bash[25056]: cluster 2023-12-07T17:17:52.250054+0000 mgr.y (mgr.24626) 103 : cluster 0 pgmap v60: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:54.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:54 smithi064 bash[36207]: cluster 2023-12-07T17:17:52.250054+0000 mgr.y (mgr.24626) 103 : cluster 0 pgmap v60: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:17:54.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495668 for osd.6 2023-12-07T17:17:54.646 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:34359738488 2023-12-07T17:17:55.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757307 got 163208757312 for osd.5 2023-12-07T17:17:55.763 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:55.811 INFO:teuthology.orchestra.run.smithi064.stdout:231928234025 2023-12-07T17:17:55.991 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:17:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:17:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:17:56.301 INFO:teuthology.orchestra.run.smithi064.stdout:77309411425 2023-12-07T17:17:56.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:56 smithi184 bash[31336]: cluster 2023-12-07T17:17:54.250529+0000 mgr.y (mgr.24626) 104 : cluster 0 pgmap v61: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:56.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:56 smithi184 bash[31336]: audit 2023-12-07T17:17:55.428019+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.64:0/180649405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T17:17:56.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:56 smithi184 bash[31336]: audit 2023-12-07T17:17:55.809055+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.64:0/659358891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T17:17:56.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[25056]: cluster 2023-12-07T17:17:54.250529+0000 mgr.y (mgr.24626) 104 : cluster 0 pgmap v61: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:56.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[25056]: audit 2023-12-07T17:17:55.428019+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.64:0/180649405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T17:17:56.331 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[25056]: audit 2023-12-07T17:17:55.809055+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.64:0/659358891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T17:17:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[36207]: cluster 2023-12-07T17:17:54.250529+0000 mgr.y (mgr.24626) 104 : cluster 0 pgmap v61: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[36207]: audit 2023-12-07T17:17:55.428019+0000 mon.a (mon.0) 812 : audit 0 from='client.? 172.21.15.64:0/180649405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T17:17:56.332 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:56 smithi064 bash[36207]: audit 2023-12-07T17:17:55.809055+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.64:0/659358891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T17:17:56.435 INFO:teuthology.orchestra.run.smithi064.stdout:55834574956 2023-12-07T17:17:56.554 INFO:teuthology.orchestra.run.smithi064.stdout:128849018955 2023-12-07T17:17:57.317 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:17:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:17:57.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:57 smithi184 bash[31336]: audit 2023-12-07T17:17:56.299081+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.64:0/163914081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T17:17:57.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:57 smithi184 bash[31336]: audit 2023-12-07T17:17:56.432772+0000 mon.c (mon.2) 71 : audit 0 from='client.? 172.21.15.64:0/2955506351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T17:17:57.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:57 smithi184 bash[31336]: audit 2023-12-07T17:17:56.552112+0000 mon.c (mon.2) 72 : audit 0 from='client.? 172.21.15.64:0/4186034431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[25056]: audit 2023-12-07T17:17:56.299081+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.64:0/163914081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[25056]: audit 2023-12-07T17:17:56.432772+0000 mon.c (mon.2) 71 : audit 0 from='client.? 172.21.15.64:0/2955506351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[25056]: audit 2023-12-07T17:17:56.552112+0000 mon.c (mon.2) 72 : audit 0 from='client.? 172.21.15.64:0/4186034431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[36207]: audit 2023-12-07T17:17:56.299081+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.64:0/163914081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[36207]: audit 2023-12-07T17:17:56.432772+0000 mon.c (mon.2) 71 : audit 0 from='client.? 172.21.15.64:0/2955506351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T17:17:57.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:57 smithi064 bash[36207]: audit 2023-12-07T17:17:56.552112+0000 mon.c (mon.2) 72 : audit 0 from='client.? 172.21.15.64:0/4186034431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T17:17:57.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738482 got 34359738488 for osd.0 2023-12-07T17:17:57.973 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:58.229 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:58 smithi064 bash[25056]: cluster 2023-12-07T17:17:56.250859+0000 mgr.y (mgr.24626) 105 : cluster 0 pgmap v62: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:58.229 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:17:58 smithi064 bash[25056]: audit 2023-12-07T17:17:57.074951+0000 mgr.y (mgr.24626) 106 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:58.229 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:58 smithi064 bash[36207]: cluster 2023-12-07T17:17:56.250859+0000 mgr.y (mgr.24626) 105 : cluster 0 pgmap v62: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:58.230 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:17:58 smithi064 bash[36207]: audit 2023-12-07T17:17:57.074951+0000 mgr.y (mgr.24626) 106 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:58.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:58 smithi184 bash[31336]: cluster 2023-12-07T17:17:56.250859+0000 mgr.y (mgr.24626) 105 : cluster 0 pgmap v62: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:17:58.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:17:58 smithi184 bash[31336]: audit 2023-12-07T17:17:57.074951+0000 mgr.y (mgr.24626) 106 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:17:58.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234019 got 231928234025 for osd.7 2023-12-07T17:17:58.421 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:58.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018950 got 128849018955 for osd.4 2023-12-07T17:17:58.926 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:59.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411420 got 77309411425 for osd.2 2023-12-07T17:17:59.557 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:59.575 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574951 got 55834574956 for osd.1 2023-12-07T17:17:59.575 DEBUG:teuthology.parallel:result is None 2023-12-07T17:17:59.575 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-07T17:17:59.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T17:18:00.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:00 smithi184 bash[31336]: cluster 2023-12-07T17:17:58.251418+0000 mgr.y (mgr.24626) 107 : cluster 0 pgmap v63: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:00.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:00 smithi064 bash[25056]: cluster 2023-12-07T17:17:58.251418+0000 mgr.y (mgr.24626) 107 : cluster 0 pgmap v63: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:00.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:00 smithi064 bash[36207]: cluster 2023-12-07T17:17:58.251418+0000 mgr.y (mgr.24626) 107 : cluster 0 pgmap v63: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:02.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:02 smithi064 bash[36207]: cluster 2023-12-07T17:18:00.252018+0000 mgr.y (mgr.24626) 108 : cluster 0 pgmap v64: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:02.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:02 smithi064 bash[25056]: cluster 2023-12-07T17:18:00.252018+0000 mgr.y (mgr.24626) 108 : cluster 0 pgmap v64: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:02.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:02 smithi184 bash[31336]: cluster 2023-12-07T17:18:00.252018+0000 mgr.y (mgr.24626) 108 : cluster 0 pgmap v64: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:04.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:04 smithi064 bash[25056]: cluster 2023-12-07T17:18:02.252808+0000 mgr.y (mgr.24626) 109 : cluster 0 pgmap v65: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:04.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:04 smithi064 bash[36207]: cluster 2023-12-07T17:18:02.252808+0000 mgr.y (mgr.24626) 109 : cluster 0 pgmap v65: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:04.392 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:04.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:04 smithi184 bash[31336]: cluster 2023-12-07T17:18:02.252808+0000 mgr.y (mgr.24626) 109 : cluster 0 pgmap v65: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:06.080 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:06.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:06 smithi064 bash[25056]: cluster 2023-12-07T17:18:04.253359+0000 mgr.y (mgr.24626) 110 : cluster 0 pgmap v66: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:06.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:06 smithi064 bash[36207]: cluster 2023-12-07T17:18:04.253359+0000 mgr.y (mgr.24626) 110 : cluster 0 pgmap v66: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:06.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:06 smithi184 bash[31336]: cluster 2023-12-07T17:18:04.253359+0000 mgr.y (mgr.24626) 110 : cluster 0 pgmap v66: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:07.356 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:07 smithi064 bash[25056]: audit 2023-12-07T17:18:06.278205+0000 mon.c (mon.2) 73 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:07.356 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:07 smithi064 bash[36207]: audit 2023-12-07T17:18:06.278205+0000 mon.c (mon.2) 73 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:07.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:07 smithi184 bash[31336]: audit 2023-12-07T17:18:06.278205+0000 mon.c (mon.2) 73 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:07.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:07.810 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:18:07.811 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-12-07T17:18:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:08 smithi064 bash[25056]: cluster 2023-12-07T17:18:06.253809+0000 mgr.y (mgr.24626) 111 : cluster 0 pgmap v67: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:08 smithi064 bash[25056]: audit 2023-12-07T17:18:07.086271+0000 mgr.y (mgr.24626) 112 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:08.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:08 smithi064 bash[36207]: cluster 2023-12-07T17:18:06.253809+0000 mgr.y (mgr.24626) 111 : cluster 0 pgmap v67: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:08.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:08 smithi064 bash[36207]: audit 2023-12-07T17:18:07.086271+0000 mgr.y (mgr.24626) 112 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:08.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:08 smithi184 bash[31336]: cluster 2023-12-07T17:18:06.253809+0000 mgr.y (mgr.24626) 111 : cluster 0 pgmap v67: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:08.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:08 smithi184 bash[31336]: audit 2023-12-07T17:18:07.086271+0000 mgr.y (mgr.24626) 112 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:08.589 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-12-07T17:18:06.253621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592232,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":417,"num_read_kb":503,"num_write":261,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":223708,"kb_used_data":7560,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749704740,"statfs":{"total":767926730752,"available":767697653760,"internally_reserved":0,"allocated":7741440,"data_stored":4303493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003324"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.961176+0000","last_change":"2023-12-07T17:15:09.745101+0000","last_active":"2023-12-07T17:16:05.961176+0000","last_peered":"2023-12-07T17:16:05.961176+0000","last_clean":"2023-12-07T17:16:05.961176+0000","last_became_active":"2023-12-07T17:15:09.744765+0000","last_became_peered":"2023-12-07T17:15:09.744765+0000","last_unstale":"2023-12-07T17:16:05.961176+0000","last_undegraded":"2023-12-07T17:16:05.961176+0000","last_fullsized":"2023-12-07T17:16:05.961176+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:21.138027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963912+0000","last_change":"2023-12-07T17:15:09.734969+0000","last_active":"2023-12-07T17:16:05.963912+0000","last_peered":"2023-12-07T17:16:05.963912+0000","last_clean":"2023-12-07T17:16:05.963912+0000","last_became_active":"2023-12-07T17:15:09.734611+0000","last_became_peered":"2023-12-07T17:15:09.734611+0000","last_unstale":"2023-12-07T17:16:05.963912+0000","last_undegraded":"2023-12-07T17:16:05.963912+0000","last_fullsized":"2023-12-07T17:16:05.963912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:53:53.170777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991308+0000","last_change":"2023-12-07T17:15:27.883869+0000","last_active":"2023-12-07T17:16:06.991308+0000","last_peered":"2023-12-07T17:16:06.991308+0000","last_clean":"2023-12-07T17:16:06.991308+0000","last_became_active":"2023-12-07T17:15:27.882890+0000","last_became_peered":"2023-12-07T17:15:27.882890+0000","last_unstale":"2023-12-07T17:16:06.991308+0000","last_undegraded":"2023-12-07T17:16:06.991308+0000","last_fullsized":"2023-12-07T17:16:06.991308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:26:22.065704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992772+0000","last_change":"2023-12-07T17:15:09.736081+0000","last_active":"2023-12-07T17:16:06.992772+0000","last_peered":"2023-12-07T17:16:06.992772+0000","last_clean":"2023-12-07T17:16:06.992772+0000","last_became_active":"2023-12-07T17:15:09.735779+0000","last_became_peered":"2023-12-07T17:15:09.735779+0000","last_unstale":"2023-12-07T17:16:06.992772+0000","last_undegraded":"2023-12-07T17:16:06.992772+0000","last_fullsized":"2023-12-07T17:16:06.992772+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:44.827805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963737+0000","last_change":"2023-12-07T17:15:09.738713+0000","last_active":"2023-12-07T17:16:05.963737+0000","last_peered":"2023-12-07T17:16:05.963737+0000","last_clean":"2023-12-07T17:16:05.963737+0000","last_became_active":"2023-12-07T17:15:09.738355+0000","last_became_peered":"2023-12-07T17:15:09.738355+0000","last_unstale":"2023-12-07T17:16:05.963737+0000","last_undegraded":"2023-12-07T17:16:05.963737+0000","last_fullsized":"2023-12-07T17:16:05.963737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:40:04.301220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491335+0000","last_change":"2023-12-07T17:15:09.737499+0000","last_active":"2023-12-07T17:16:06.491335+0000","last_peered":"2023-12-07T17:16:06.491335+0000","last_clean":"2023-12-07T17:16:06.491335+0000","last_became_active":"2023-12-07T17:15:09.737120+0000","last_became_peered":"2023-12-07T17:15:09.737120+0000","last_unstale":"2023-12-07T17:16:06.491335+0000","last_undegraded":"2023-12-07T17:16:06.491335+0000","last_fullsized":"2023-12-07T17:16:06.491335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:49:03.869434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"60'2","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963365+0000","last_change":"2023-12-07T17:15:09.740336+0000","last_active":"2023-12-07T17:16:05.963365+0000","last_peered":"2023-12-07T17:16:05.963365+0000","last_clean":"2023-12-07T17:16:05.963365+0000","last_became_active":"2023-12-07T17:15:09.738682+0000","last_became_peered":"2023-12-07T17:15:09.738682+0000","last_unstale":"2023-12-07T17:16:05.963365+0000","last_undegraded":"2023-12-07T17:16:05.963365+0000","last_fullsized":"2023-12-07T17:16:05.963365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:24:46.013812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991852+0000","last_change":"2023-12-07T17:15:09.735145+0000","last_active":"2023-12-07T17:16:06.991852+0000","last_peered":"2023-12-07T17:16:06.991852+0000","last_clean":"2023-12-07T17:16:06.991852+0000","last_became_active":"2023-12-07T17:15:09.734810+0000","last_became_peered":"2023-12-07T17:15:09.734810+0000","last_unstale":"2023-12-07T17:16:06.991852+0000","last_undegraded":"2023-12-07T17:16:06.991852+0000","last_fullsized":"2023-12-07T17:16:06.991852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:15:21.343209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491549+0000","last_change":"2023-12-07T17:15:09.746232+0000","last_active":"2023-12-07T17:16:06.491549+0000","last_peered":"2023-12-07T17:16:06.491549+0000","last_clean":"2023-12-07T17:16:06.491549+0000","last_became_active":"2023-12-07T17:15:09.745401+0000","last_became_peered":"2023-12-07T17:15:09.745401+0000","last_unstale":"2023-12-07T17:16:06.491549+0000","last_undegraded":"2023-12-07T17:16:06.491549+0000","last_fullsized":"2023-12-07T17:16:06.491549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T03:19:50.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992994+0000","last_change":"2023-12-07T17:15:09.739107+0000","last_active":"2023-12-07T17:16:06.992994+0000","last_peered":"2023-12-07T17:16:06.992994+0000","last_clean":"2023-12-07T17:16:06.992994+0000","last_became_active":"2023-12-07T17:15:09.738758+0000","last_became_peered":"2023-12-07T17:15:09.738758+0000","last_unstale":"2023-12-07T17:16:06.992994+0000","last_undegraded":"2023-12-07T17:16:06.992994+0000","last_fullsized":"2023-12-07T17:16:06.992994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T05:12:22.803108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140697+0000","last_change":"2023-12-07T17:15:09.740165+0000","last_active":"2023-12-07T17:16:06.140697+0000","last_peered":"2023-12-07T17:16:06.140697+0000","last_clean":"2023-12-07T17:16:06.140697+0000","last_became_active":"2023-12-07T17:15:09.739820+0000","last_became_peered":"2023-12-07T17:15:09.739820+0000","last_unstale":"2023-12-07T17:16:06.140697+0000","last_undegraded":"2023-12-07T17:16:06.140697+0000","last_fullsized":"2023-12-07T17:16:06.140697+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T02:44:16.243037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491302+0000","last_change":"2023-12-07T17:15:09.742176+0000","last_active":"2023-12-07T17:16:06.491302+0000","last_peered":"2023-12-07T17:16:06.491302+0000","last_clean":"2023-12-07T17:16:06.491302+0000","last_became_active":"2023-12-07T17:15:09.738366+0000","last_became_peered":"2023-12-07T17:15:09.738366+0000","last_unstale":"2023-12-07T17:16:06.491302+0000","last_undegraded":"2023-12-07T17:16:06.491302+0000","last_fullsized":"2023-12-07T17:16:06.491302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:22:17.951815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.960759+0000","last_change":"2023-12-07T17:15:09.742056+0000","last_active":"2023-12-07T17:16:05.960759+0000","last_peered":"2023-12-07T17:16:05.960759+0000","last_clean":"2023-12-07T17:16:05.960759+0000","last_became_active":"2023-12-07T17:15:09.741708+0000","last_became_peered":"2023-12-07T17:15:09.741708+0000","last_unstale":"2023-12-07T17:16:05.960759+0000","last_undegraded":"2023-12-07T17:16:05.960759+0000","last_fullsized":"2023-12-07T17:16:05.960759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:52:12.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992259+0000","last_change":"2023-12-07T17:15:09.735452+0000","last_active":"2023-12-07T17:16:06.992259+0000","last_peered":"2023-12-07T17:16:06.992259+0000","last_clean":"2023-12-07T17:16:06.992259+0000","last_became_active":"2023-12-07T17:15:09.734922+0000","last_became_peered":"2023-12-07T17:15:09.734922+0000","last_unstale":"2023-12-07T17:16:06.992259+0000","last_undegraded":"2023-12-07T17:16:06.992259+0000","last_fullsized":"2023-12-07T17:16:06.992259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T05:07:43.090081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491141+0000","last_change":"2023-12-07T17:15:09.734421+0000","last_active":"2023-12-07T17:16:06.491141+0000","last_peered":"2023-12-07T17:16:06.491141+0000","last_clean":"2023-12-07T17:16:06.491141+0000","last_became_active":"2023-12-07T17:15:09.734134+0000","last_became_peered":"2023-12-07T17:15:09.734134+0000","last_unstale":"2023-12-07T17:16:06.491141+0000","last_undegraded":"2023-12-07T17:16:06.491141+0000","last_fullsized":"2023-12-07T17:16:06.491141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:39.069901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991187+0000","last_change":"2023-12-07T17:15:09.738156+0000","last_active":"2023-12-07T17:16:06.991187+0000","last_peered":"2023-12-07T17:16:06.991187+0000","last_clean":"2023-12-07T17:16:06.991187+0000","last_became_active":"2023-12-07T17:15:09.737738+0000","last_became_peered":"2023-12-07T17:15:09.737738+0000","last_unstale":"2023-12-07T17:16:06.991187+0000","last_undegraded":"2023-12-07T17:16:06.991187+0000","last_fullsized":"2023-12-07T17:16:06.991187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:18:49.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.962500+0000","last_change":"2023-12-07T17:15:09.740051+0000","last_active":"2023-12-07T17:16:05.962500+0000","last_peered":"2023-12-07T17:16:05.962500+0000","last_clean":"2023-12-07T17:16:05.962500+0000","last_became_active":"2023-12-07T17:15:09.739752+0000","last_became_peered":"2023-12-07T17:15:09.739752+0000","last_unstale":"2023-12-07T17:16:05.962500+0000","last_undegraded":"2023-12-07T17:16:05.962500+0000","last_fullsized":"2023-12-07T17:16:05.962500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:35:51.955817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140462+0000","last_change":"2023-12-07T17:15:09.740920+0000","last_active":"2023-12-07T17:16:06.140462+0000","last_peered":"2023-12-07T17:16:06.140462+0000","last_clean":"2023-12-07T17:16:06.140462+0000","last_became_active":"2023-12-07T17:15:09.740628+0000","last_became_peered":"2023-12-07T17:15:09.740628+0000","last_unstale":"2023-12-07T17:16:06.140462+0000","last_undegraded":"2023-12-07T17:16:06.140462+0000","last_fullsized":"2023-12-07T17:16:06.140462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T02:34:32.982853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992067+0000","last_change":"2023-12-07T17:15:09.739301+0000","last_active":"2023-12-07T17:16:06.992067+0000","last_peered":"2023-12-07T17:16:06.992067+0000","last_clean":"2023-12-07T17:16:06.992067+0000","last_became_active":"2023-12-07T17:15:09.738819+0000","last_became_peered":"2023-12-07T17:15:09.738819+0000","last_unstale":"2023-12-07T17:16:06.992067+0000","last_undegraded":"2023-12-07T17:16:06.992067+0000","last_fullsized":"2023-12-07T17:16:06.992067+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:27:01.628578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"70'5","reported_seq":182,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:18:00.457066+0000","last_change":"2023-12-07T17:15:22.723013+0000","last_active":"2023-12-07T17:18:00.457066+0000","last_peered":"2023-12-07T17:18:00.457066+0000","last_clean":"2023-12-07T17:18:00.457066+0000","last_became_active":"2023-12-07T17:15:13.560069+0000","last_became_peered":"2023-12-07T17:15:13.560069+0000","last_unstale":"2023-12-07T17:18:00.457066+0000","last_undegraded":"2023-12-07T17:18:00.457066+0000","last_fullsized":"2023-12-07T17:18:00.457066+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:25:18.360736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073499100000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":151,"num_read_kb":146,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991147+0000","last_change":"2023-12-07T17:15:09.738077+0000","last_active":"2023-12-07T17:16:06.991147+0000","last_peered":"2023-12-07T17:16:06.991147+0000","last_clean":"2023-12-07T17:16:06.991147+0000","last_became_active":"2023-12-07T17:15:09.737703+0000","last_became_peered":"2023-12-07T17:15:09.737703+0000","last_unstale":"2023-12-07T17:16:06.991147+0000","last_undegraded":"2023-12-07T17:16:06.991147+0000","last_fullsized":"2023-12-07T17:16:06.991147+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:17:16.761902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"67'2","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963453+0000","last_change":"2023-12-07T17:15:22.723394+0000","last_active":"2023-12-07T17:16:05.963453+0000","last_peered":"2023-12-07T17:16:05.963453+0000","last_clean":"2023-12-07T17:16:05.963453+0000","last_became_active":"2023-12-07T17:15:13.555197+0000","last_became_peered":"2023-12-07T17:15:13.555197+0000","last_unstale":"2023-12-07T17:16:05.963453+0000","last_undegraded":"2023-12-07T17:16:05.963453+0000","last_fullsized":"2023-12-07T17:16:05.963453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:56:25.588179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682451,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992504+0000","last_change":"2023-12-07T17:15:09.733600+0000","last_active":"2023-12-07T17:16:06.992504+0000","last_peered":"2023-12-07T17:16:06.992504+0000","last_clean":"2023-12-07T17:16:06.992504+0000","last_became_active":"2023-12-07T17:15:09.733223+0000","last_became_peered":"2023-12-07T17:15:09.733223+0000","last_unstale":"2023-12-07T17:16:06.992504+0000","last_undegraded":"2023-12-07T17:16:06.992504+0000","last_fullsized":"2023-12-07T17:16:06.992504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:14:33.913419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'188","reported_seq":331,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:08.134222+0000","last_change":"2023-12-07T17:14:32.745381+0000","last_active":"2023-12-07T17:16:08.134222+0000","last_peered":"2023-12-07T17:16:08.134222+0000","last_clean":"2023-12-07T17:16:08.134222+0000","last_became_active":"2023-12-07T17:14:32.744449+0000","last_became_peered":"2023-12-07T17:14:32.744449+0000","last_unstale":"2023-12-07T17:16:08.134222+0000","last_undegraded":"2023-12-07T17:16:08.134222+0000","last_fullsized":"2023-12-07T17:16:08.134222+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:09:56.938364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:09:56.938364+0000","last_clean_scrub_stamp":"2023-12-07T17:09:56.938364+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:45:58.588219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.960311+0000","last_change":"2023-12-07T17:15:22.953440+0000","last_active":"2023-12-07T17:16:05.960311+0000","last_peered":"2023-12-07T17:16:05.960311+0000","last_clean":"2023-12-07T17:16:05.960311+0000","last_became_active":"2023-12-07T17:15:13.216515+0000","last_became_peered":"2023-12-07T17:15:13.216515+0000","last_unstale":"2023-12-07T17:16:05.960311+0000","last_undegraded":"2023-12-07T17:16:05.960311+0000","last_fullsized":"2023-12-07T17:16:05.960311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:51:41.566664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060667900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992183+0000","last_change":"2023-12-07T17:15:09.743429+0000","last_active":"2023-12-07T17:16:06.992183+0000","last_peered":"2023-12-07T17:16:06.992183+0000","last_clean":"2023-12-07T17:16:06.992183+0000","last_became_active":"2023-12-07T17:15:09.742733+0000","last_became_peered":"2023-12-07T17:15:09.742733+0000","last_unstale":"2023-12-07T17:16:06.992183+0000","last_undegraded":"2023-12-07T17:16:06.992183+0000","last_fullsized":"2023-12-07T17:16:06.992183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:24.179470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992033+0000","last_change":"2023-12-07T17:15:09.744152+0000","last_active":"2023-12-07T17:16:06.992033+0000","last_peered":"2023-12-07T17:16:06.992033+0000","last_clean":"2023-12-07T17:16:06.992033+0000","last_became_active":"2023-12-07T17:15:09.743832+0000","last_became_peered":"2023-12-07T17:15:09.743832+0000","last_unstale":"2023-12-07T17:16:06.992033+0000","last_undegraded":"2023-12-07T17:16:06.992033+0000","last_fullsized":"2023-12-07T17:16:06.992033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T03:38:51.956613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"60'2","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140478+0000","last_change":"2023-12-07T17:15:09.732781+0000","last_active":"2023-12-07T17:16:06.140478+0000","last_peered":"2023-12-07T17:16:06.140478+0000","last_clean":"2023-12-07T17:16:06.140478+0000","last_became_active":"2023-12-07T17:15:09.731586+0000","last_became_peered":"2023-12-07T17:15:09.731586+0000","last_unstale":"2023-12-07T17:16:06.140478+0000","last_undegraded":"2023-12-07T17:16:06.140478+0000","last_fullsized":"2023-12-07T17:16:06.140478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:54.633492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491549+0000","last_change":"2023-12-07T17:15:09.731572+0000","last_active":"2023-12-07T17:16:06.491549+0000","last_peered":"2023-12-07T17:16:06.491549+0000","last_clean":"2023-12-07T17:16:06.491549+0000","last_became_active":"2023-12-07T17:15:09.731239+0000","last_became_peered":"2023-12-07T17:15:09.731239+0000","last_unstale":"2023-12-07T17:16:06.491549+0000","last_undegraded":"2023-12-07T17:16:06.491549+0000","last_fullsized":"2023-12-07T17:16:06.491549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:02:56.787357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991782+0000","last_change":"2023-12-07T17:15:09.743613+0000","last_active":"2023-12-07T17:16:06.991782+0000","last_peered":"2023-12-07T17:16:06.991782+0000","last_clean":"2023-12-07T17:16:06.991782+0000","last_became_active":"2023-12-07T17:15:09.743141+0000","last_became_peered":"2023-12-07T17:15:09.743141+0000","last_unstale":"2023-12-07T17:16:06.991782+0000","last_undegraded":"2023-12-07T17:16:06.991782+0000","last_fullsized":"2023-12-07T17:16:06.991782+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:16:31.499043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140862+0000","last_change":"2023-12-07T17:15:09.729655+0000","last_active":"2023-12-07T17:16:06.140862+0000","last_peered":"2023-12-07T17:16:06.140862+0000","last_clean":"2023-12-07T17:16:06.140862+0000","last_became_active":"2023-12-07T17:15:09.728138+0000","last_became_peered":"2023-12-07T17:15:09.728138+0000","last_unstale":"2023-12-07T17:16:06.140862+0000","last_undegraded":"2023-12-07T17:16:06.140862+0000","last_fullsized":"2023-12-07T17:16:06.140862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T01:48:51.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140548+0000","last_change":"2023-12-07T17:15:09.731496+0000","last_active":"2023-12-07T17:16:06.140548+0000","last_peered":"2023-12-07T17:16:06.140548+0000","last_clean":"2023-12-07T17:16:06.140548+0000","last_became_active":"2023-12-07T17:15:09.731025+0000","last_became_peered":"2023-12-07T17:15:09.731025+0000","last_unstale":"2023-12-07T17:16:06.140548+0000","last_undegraded":"2023-12-07T17:16:06.140548+0000","last_fullsized":"2023-12-07T17:16:06.140548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:35:33.108346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992317+0000","last_change":"2023-12-07T17:15:09.733583+0000","last_active":"2023-12-07T17:16:06.992317+0000","last_peered":"2023-12-07T17:16:06.992317+0000","last_clean":"2023-12-07T17:16:06.992317+0000","last_became_active":"2023-12-07T17:15:09.733241+0000","last_became_peered":"2023-12-07T17:15:09.733241+0000","last_unstale":"2023-12-07T17:16:06.992317+0000","last_undegraded":"2023-12-07T17:16:06.992317+0000","last_fullsized":"2023-12-07T17:16:06.992317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:13:59.244553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991284+0000","last_change":"2023-12-07T17:15:09.738066+0000","last_active":"2023-12-07T17:16:06.991284+0000","last_peered":"2023-12-07T17:16:06.991284+0000","last_clean":"2023-12-07T17:16:06.991284+0000","last_became_active":"2023-12-07T17:15:09.737689+0000","last_became_peered":"2023-12-07T17:15:09.737689+0000","last_unstale":"2023-12-07T17:16:06.991284+0000","last_undegraded":"2023-12-07T17:16:06.991284+0000","last_fullsized":"2023-12-07T17:16:06.991284+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:23:23.277498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140561+0000","last_change":"2023-12-07T17:15:09.732745+0000","last_active":"2023-12-07T17:16:06.140561+0000","last_peered":"2023-12-07T17:16:06.140561+0000","last_clean":"2023-12-07T17:16:06.140561+0000","last_became_active":"2023-12-07T17:15:09.732219+0000","last_became_peered":"2023-12-07T17:15:09.732219+0000","last_unstale":"2023-12-07T17:16:06.140561+0000","last_undegraded":"2023-12-07T17:16:06.140561+0000","last_fullsized":"2023-12-07T17:16:06.140561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:14:10.820659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991338+0000","last_change":"2023-12-07T17:15:09.731637+0000","last_active":"2023-12-07T17:16:06.991338+0000","last_peered":"2023-12-07T17:16:06.991338+0000","last_clean":"2023-12-07T17:16:06.991338+0000","last_became_active":"2023-12-07T17:15:09.731212+0000","last_became_peered":"2023-12-07T17:15:09.731212+0000","last_unstale":"2023-12-07T17:16:06.991338+0000","last_undegraded":"2023-12-07T17:16:06.991338+0000","last_fullsized":"2023-12-07T17:16:06.991338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:22:45.616688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28316,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712740,"statfs":{"total":95990841344,"available":95961845760,"internally_reserved":0,"allocated":1331200,"data_stored":906020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97999999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1819999999999999,"5min":1.409,"15min":1.409},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96299999999999997,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93700000000000006,"15min":0.94699999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0089999999999999,"15min":1.0289999999999999},"min":{"1min":0.87,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.877}]},{"osd":2,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95499999999999996,"15min":0.96599999999999997},"min":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.361,"5min":1.391,"15min":1.391},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95899999999999996,"15min":0.98499999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.137,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.099}]},{"osd":3,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.98899999999999999,"15min":1.0089999999999999},"min":{"1min":0.78600000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.22,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.042,"5min":1.034,"15min":1.048},"min":{"1min":0.77600000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.417,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88800000000000001,"15min":0.91800000000000004},"min":{"1min":0.66900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.99199999999999999,"5min":1.262,"15min":1.262},"last":1.026}]},{"osd":5,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.57699999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.196},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.58299999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.4610000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1599999999999999}]},{"osd":6,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.329,"5min":1.526,"15min":1.526},"last":1.266},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.121,"15min":1.137},"min":{"1min":0.68200000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5069999999999999,"5min":1.536,"15min":1.536},"last":1.24}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28336,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712720,"statfs":{"total":95990841344,"available":95961825280,"internally_reserved":0,"allocated":1351680,"data_stored":908031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90200000000000002},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3500000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89700000000000002,"15min":0.91300000000000003},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.167,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.96599999999999997}]},{"osd":1,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.93100000000000005,"15min":0.97399999999999998},"min":{"1min":0.60399999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.113,"5min":1.704,"15min":1.704},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.878},"min":{"1min":0.58399999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":1.2889999999999999}]},{"osd":2,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.69099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96899999999999997,"15min":0.998},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.2989999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.94799999999999995,"15min":0.94599999999999995},"min":{"1min":0.69399999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.286,"5min":1.623,"15min":1.623},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.036,"15min":1.0640000000000001},"min":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4379999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.236}]},{"osd":4,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.875,"15min":0.83899999999999997},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.373,"5min":1.403,"15min":1.403},"last":0.877},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86499999999999999,"15min":0.80300000000000005},"min":{"1min":0.63800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.494,"15min":1.494},"last":1.137}]},{"osd":5,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92800000000000005,"15min":0.89400000000000002},"min":{"1min":0.68400000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.099},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.92900000000000005},"min":{"1min":0.64700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.343,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.069,"15min":1.097},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3049999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.079,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.591,"5min":1.772,"15min":1.772},"last":1.3380000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713308,"statfs":{"total":95990841344,"available":95962427392,"internally_reserved":0,"allocated":745472,"data_stored":316724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.60999999999999999},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.30199999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.67700000000000005},"min":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.34100000000000003},"max":{"1min":1.2390000000000001,"5min":1.429,"15min":1.429},"last":0.498}]},{"osd":2,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76800000000000002,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.40000000000000002},"max":{"1min":1.1599999999999999,"5min":1.177,"15min":1.177},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.82499999999999996}]},{"osd":3,"last update":"Thu Dec 7 17:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.84999999999999998,"15min":0.82199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91300000000000003,"15min":0.87},"min":{"1min":0.61499999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Dec 7 17:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.034,"15min":0.97299999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.524,"5min":1.54,"15min":1.54},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0640000000000001,"15min":1.002},"min":{"1min":0.745,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3720000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0249999999999999,"15min":1.02},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.169,"5min":1.716,"15min":1.716},"last":0.873},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.155,"15min":1.1399999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4610000000000001,"5min":1.736,"15min":1.736},"last":1.04}]},{"osd":6,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1910000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.337,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.046,"5min":1.2370000000000001,"15min":1.292},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.248,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.246,"15min":1.2589999999999999},"min":{"1min":0.88,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.5880000000000001,"5min":1.639,"15min":1.639},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.23,"15min":1.258},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.425,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.96799999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28328,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712728,"statfs":{"total":95990841344,"available":95961833472,"internally_reserved":0,"allocated":1343488,"data_stored":907271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.751,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.36899999999999999},"max":{"1min":0.91800000000000004,"5min":2.085,"15min":2.085},"last":0.501},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.68600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.34100000000000003},"max":{"1min":1.236,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Dec 7 17:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83199999999999996,"15min":0.70699999999999996},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.39900000000000002},"max":{"1min":1.1060000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.71999999999999997},"min":{"1min":0.50600000000000001,"5min":0.42399999999999999,"15min":0.41599999999999998},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Dec 7 17:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.93600000000000005,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0449999999999999,"5min":3.25,"15min":3.25},"last":0.63}]},{"osd":4,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":0.98799999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1279999999999999,"5min":1.708,"15min":1.708},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.0629999999999999,"15min":1.032},"min":{"1min":0.626,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.204,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.1970000000000001}]},{"osd":5,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.085,"15min":1.101},"min":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2490000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.1120000000000001,"15min":1.093},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.327,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1180000000000001}]},{"osd":6,"last update":"Thu Dec 7 17:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3080000000000001,"5min":2.786,"15min":2.786},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1970000000000001,"15min":1.2250000000000001},"min":{"1min":0.88600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.496,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.0089999999999999}]},{"osd":7,"last update":"Thu Dec 7 17:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.268,"15min":1.302},"min":{"1min":0.84699999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.601,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.224,"15min":1.264},"min":{"1min":0.71199999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.7070000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.087}]}]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713316,"statfs":{"total":95990841344,"available":95962435584,"internally_reserved":0,"allocated":737280,"data_stored":316280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.76400000000000001,"15min":0.65800000000000003},"min":{"1min":0.51000000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.72299999999999998},"min":{"1min":0.496,"5min":0.48099999999999998,"15min":0.40500000000000003},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.153}]},{"osd":1,"last update":"Thu Dec 7 17:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.84399999999999997,"15min":0.68000000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.374},"max":{"1min":1.1830000000000001,"5min":1.226,"15min":1.226},"last":1.226},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.93100000000000005,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.55800000000000005,"15min":0.36899999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.86299999999999999}]},{"osd":3,"last update":"Thu Dec 7 17:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93600000000000005,"15min":0.874},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.323,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.2869999999999999}]},{"osd":4,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96599999999999997,"15min":0.93799999999999994},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.131,"15min":1.075},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3089999999999999,"5min":1.948,"15min":1.948},"last":1.3500000000000001}]},{"osd":5,"last update":"Thu Dec 7 17:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.1240000000000001,"15min":1.1399999999999999},"min":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4650000000000001,"5min":1.673,"15min":1.673},"last":1.3999999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0249999999999999,"15min":1.0409999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.381,"5min":1.784,"15min":1.784},"last":1.2010000000000001}]},{"osd":6,"last update":"Thu Dec 7 17:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.196,"15min":1.224},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.367,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.456},{"interface":"front","average":{"1min":1.091,"5min":1.206,"15min":1.2050000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.764,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.5569999999999999}]},{"osd":7,"last update":"Thu Dec 7 17:17:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.129,"15min":1.161},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4239999999999999,"5min":1.899,"15min":1.899},"last":1.524},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.3180000000000001,"15min":1.3799999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8520000000000001,"5min":2.133,"15min":2.133},"last":1.494}]}]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713308,"statfs":{"total":95990841344,"available":95962427392,"internally_reserved":0,"allocated":745472,"data_stored":316830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.68100000000000005},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.171,"5min":1.306,"15min":1.306},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.52700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.25,"15min":1.25},"last":1.014}]},{"osd":1,"last update":"Thu Dec 7 17:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.84399999999999997,"15min":0.80100000000000005},"min":{"1min":0.47299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80500000000000005,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0069999999999999,"5min":1.625,"15min":1.625},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Dec 7 17:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.879},"min":{"1min":0.64300000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1819999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.88900000000000001,"15min":0.85299999999999998},"min":{"1min":0.54900000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.647,"15min":1.647},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Dec 7 17:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":0.97899999999999998},"min":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4890000000000001,"5min":1.522,"15min":1.522},"last":1.054},{"interface":"front","average":{"1min":1.022,"5min":1.091,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.353,"5min":1.829,"15min":1.829},"last":1.2470000000000001}]},{"osd":5,"last update":"Thu Dec 7 17:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1100000000000001,"15min":1.0780000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5569999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.425},{"interface":"front","average":{"1min":1.016,"5min":1.103,"15min":1.0860000000000001},"min":{"1min":0.68600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3460000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.3560000000000001}]},{"osd":6,"last update":"Thu Dec 7 17:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1830000000000001,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.655,"5min":1.927,"15min":1.927},"last":1.319},{"interface":"front","average":{"1min":1.093,"5min":1.1619999999999999,"15min":1.1719999999999999},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5049999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.4690000000000001}]},{"osd":7,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.216,"15min":1.244},"min":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5840000000000001,"5min":1.875,"15min":1.875},"last":1.292},{"interface":"front","average":{"1min":1.073,"5min":1.151,"15min":1.157},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.544,"5min":1.583,"15min":1.583},"last":1.512}]}]},{"osd":4,"up_from":30,"seq":128849018957,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713304,"statfs":{"total":95990841344,"available":95962423296,"internally_reserved":0,"allocated":749568,"data_stored":316427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.86199999999999999},"min":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"max":{"1min":1.097,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.034},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.89900000000000002,"15min":0.92500000000000004},"min":{"1min":0.50600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.91400000000000003,"15min":0.88800000000000001},"min":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.204,"5min":1.425,"15min":1.425},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93100000000000005,"15min":0.93999999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.093}]},{"osd":2,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93400000000000005,"15min":0.94999999999999996},"min":{"1min":0.66700000000000004,"5min":0.63,"15min":0.63},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3049999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.222}]},{"osd":3,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0029999999999999,"15min":1.0009999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3580000000000001,"5min":1.385,"15min":1.385},"last":1.272}]},{"osd":5,"last update":"Thu Dec 7 17:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85899999999999999,"15min":0.77300000000000002},"min":{"1min":0.50700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.98399999999999999,"15min":0.90600000000000003},"min":{"1min":0.56100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.383,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Thu Dec 7 17:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.296,"5min":1.581,"15min":1.581},"last":1.179},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94199999999999995,"15min":0.89600000000000002},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4139999999999999,"5min":1.448,"15min":1.448},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Dec 7 17:17:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.052},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4419999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.008,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4330000000000001,"5min":1.536,"15min":1.536},"last":1.151}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713316,"statfs":{"total":95990841344,"available":95962435584,"internally_reserved":0,"allocated":737280,"data_stored":315910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.91800000000000004,"15min":0.89300000000000002},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.125,"5min":1.464,"15min":1.464},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.91400000000000003},"min":{"1min":0.62,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1639999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":1.2789999999999999}]},{"osd":1,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.92100000000000004},"min":{"1min":0.73499999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1579999999999999,"5min":1.286,"15min":1.286},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.89100000000000001,"15min":0.86499999999999999},"min":{"1min":0.56100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.155,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97499999999999998,"15min":0.89500000000000002},"min":{"1min":0.81100000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0009999999999999,"15min":0.96199999999999997},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97199999999999998,"15min":0.94199999999999995},"min":{"1min":0.67900000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0009999999999999},"min":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.2}]},{"osd":4,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.82899999999999996,"15min":0.76800000000000002},"min":{"1min":0.58199999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.179,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.161},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91200000000000003,"15min":0.86099999999999999},"min":{"1min":0.54200000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.123}]},{"osd":6,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89200000000000002,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1859999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":0.97599999999999998},"min":{"1min":0.68400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.294,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2569999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.343},{"interface":"front","average":{"1min":1,"5min":1.0860000000000001,"15min":1.1180000000000001},"min":{"1min":0.72599999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2849999999999999,"5min":1.51,"15min":1.51},"last":1.4239999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T17:18:08.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T17:18:09.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:09 smithi064 bash[36207]: audit 2023-12-07T17:18:07.799294+0000 mgr.y (mgr.24626) 113 : audit 0 from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:09.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:09 smithi064 bash[25056]: audit 2023-12-07T17:18:07.799294+0000 mgr.y (mgr.24626) 113 : audit 0 from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:09.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:09 smithi184 bash[31336]: audit 2023-12-07T17:18:07.799294+0000 mgr.y (mgr.24626) 113 : audit 0 from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:10.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:10 smithi064 bash[25056]: cluster 2023-12-07T17:18:08.254609+0000 mgr.y (mgr.24626) 114 : cluster 0 pgmap v68: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:10.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:10 smithi064 bash[36207]: cluster 2023-12-07T17:18:08.254609+0000 mgr.y (mgr.24626) 114 : cluster 0 pgmap v68: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:10.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:10 smithi184 bash[31336]: cluster 2023-12-07T17:18:08.254609+0000 mgr.y (mgr.24626) 114 : cluster 0 pgmap v68: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:12.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:12 smithi064 bash[25056]: cluster 2023-12-07T17:18:10.255250+0000 mgr.y (mgr.24626) 115 : cluster 0 pgmap v69: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:12.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:12 smithi064 bash[36207]: cluster 2023-12-07T17:18:10.255250+0000 mgr.y (mgr.24626) 115 : cluster 0 pgmap v69: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:12.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:12 smithi184 bash[31336]: cluster 2023-12-07T17:18:10.255250+0000 mgr.y (mgr.24626) 115 : cluster 0 pgmap v69: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:13.365 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:14.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:14 smithi064 bash[25056]: cluster 2023-12-07T17:18:12.256169+0000 mgr.y (mgr.24626) 116 : cluster 0 pgmap v70: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:14.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:14 smithi064 bash[36207]: cluster 2023-12-07T17:18:12.256169+0000 mgr.y (mgr.24626) 116 : cluster 0 pgmap v70: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:14.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:14 smithi184 bash[31336]: cluster 2023-12-07T17:18:12.256169+0000 mgr.y (mgr.24626) 116 : cluster 0 pgmap v70: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:15.972 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:16.355 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:18:16.357 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-12-07T17:18:16.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:16 smithi064 bash[25056]: cluster 2023-12-07T17:18:14.256802+0000 mgr.y (mgr.24626) 117 : cluster 0 pgmap v71: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:16.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:16 smithi064 bash[36207]: cluster 2023-12-07T17:18:14.256802+0000 mgr.y (mgr.24626) 117 : cluster 0 pgmap v71: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:16.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:16 smithi184 bash[31336]: cluster 2023-12-07T17:18:14.256802+0000 mgr.y (mgr.24626) 117 : cluster 0 pgmap v71: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:17.184 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-12-07T17:18:16.257095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592232,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":513,"num_write":261,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":223708,"kb_used_data":7560,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749704740,"statfs":{"total":767926730752,"available":767697653760,"internally_reserved":0,"allocated":7741440,"data_stored":4303493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004064"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.961176+0000","last_change":"2023-12-07T17:15:09.745101+0000","last_active":"2023-12-07T17:16:05.961176+0000","last_peered":"2023-12-07T17:16:05.961176+0000","last_clean":"2023-12-07T17:16:05.961176+0000","last_became_active":"2023-12-07T17:15:09.744765+0000","last_became_peered":"2023-12-07T17:15:09.744765+0000","last_unstale":"2023-12-07T17:16:05.961176+0000","last_undegraded":"2023-12-07T17:16:05.961176+0000","last_fullsized":"2023-12-07T17:16:05.961176+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:21.138027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963912+0000","last_change":"2023-12-07T17:15:09.734969+0000","last_active":"2023-12-07T17:16:05.963912+0000","last_peered":"2023-12-07T17:16:05.963912+0000","last_clean":"2023-12-07T17:16:05.963912+0000","last_became_active":"2023-12-07T17:15:09.734611+0000","last_became_peered":"2023-12-07T17:15:09.734611+0000","last_unstale":"2023-12-07T17:16:05.963912+0000","last_undegraded":"2023-12-07T17:16:05.963912+0000","last_fullsized":"2023-12-07T17:16:05.963912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:53:53.170777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991308+0000","last_change":"2023-12-07T17:15:27.883869+0000","last_active":"2023-12-07T17:16:06.991308+0000","last_peered":"2023-12-07T17:16:06.991308+0000","last_clean":"2023-12-07T17:16:06.991308+0000","last_became_active":"2023-12-07T17:15:27.882890+0000","last_became_peered":"2023-12-07T17:15:27.882890+0000","last_unstale":"2023-12-07T17:16:06.991308+0000","last_undegraded":"2023-12-07T17:16:06.991308+0000","last_fullsized":"2023-12-07T17:16:06.991308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:26:22.065704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992772+0000","last_change":"2023-12-07T17:15:09.736081+0000","last_active":"2023-12-07T17:16:06.992772+0000","last_peered":"2023-12-07T17:16:06.992772+0000","last_clean":"2023-12-07T17:16:06.992772+0000","last_became_active":"2023-12-07T17:15:09.735779+0000","last_became_peered":"2023-12-07T17:15:09.735779+0000","last_unstale":"2023-12-07T17:16:06.992772+0000","last_undegraded":"2023-12-07T17:16:06.992772+0000","last_fullsized":"2023-12-07T17:16:06.992772+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:44.827805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963737+0000","last_change":"2023-12-07T17:15:09.738713+0000","last_active":"2023-12-07T17:16:05.963737+0000","last_peered":"2023-12-07T17:16:05.963737+0000","last_clean":"2023-12-07T17:16:05.963737+0000","last_became_active":"2023-12-07T17:15:09.738355+0000","last_became_peered":"2023-12-07T17:15:09.738355+0000","last_unstale":"2023-12-07T17:16:05.963737+0000","last_undegraded":"2023-12-07T17:16:05.963737+0000","last_fullsized":"2023-12-07T17:16:05.963737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:40:04.301220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491335+0000","last_change":"2023-12-07T17:15:09.737499+0000","last_active":"2023-12-07T17:16:06.491335+0000","last_peered":"2023-12-07T17:16:06.491335+0000","last_clean":"2023-12-07T17:16:06.491335+0000","last_became_active":"2023-12-07T17:15:09.737120+0000","last_became_peered":"2023-12-07T17:15:09.737120+0000","last_unstale":"2023-12-07T17:16:06.491335+0000","last_undegraded":"2023-12-07T17:16:06.491335+0000","last_fullsized":"2023-12-07T17:16:06.491335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:49:03.869434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"60'2","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963365+0000","last_change":"2023-12-07T17:15:09.740336+0000","last_active":"2023-12-07T17:16:05.963365+0000","last_peered":"2023-12-07T17:16:05.963365+0000","last_clean":"2023-12-07T17:16:05.963365+0000","last_became_active":"2023-12-07T17:15:09.738682+0000","last_became_peered":"2023-12-07T17:15:09.738682+0000","last_unstale":"2023-12-07T17:16:05.963365+0000","last_undegraded":"2023-12-07T17:16:05.963365+0000","last_fullsized":"2023-12-07T17:16:05.963365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:24:46.013812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991852+0000","last_change":"2023-12-07T17:15:09.735145+0000","last_active":"2023-12-07T17:16:06.991852+0000","last_peered":"2023-12-07T17:16:06.991852+0000","last_clean":"2023-12-07T17:16:06.991852+0000","last_became_active":"2023-12-07T17:15:09.734810+0000","last_became_peered":"2023-12-07T17:15:09.734810+0000","last_unstale":"2023-12-07T17:16:06.991852+0000","last_undegraded":"2023-12-07T17:16:06.991852+0000","last_fullsized":"2023-12-07T17:16:06.991852+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:15:21.343209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491549+0000","last_change":"2023-12-07T17:15:09.746232+0000","last_active":"2023-12-07T17:16:06.491549+0000","last_peered":"2023-12-07T17:16:06.491549+0000","last_clean":"2023-12-07T17:16:06.491549+0000","last_became_active":"2023-12-07T17:15:09.745401+0000","last_became_peered":"2023-12-07T17:15:09.745401+0000","last_unstale":"2023-12-07T17:16:06.491549+0000","last_undegraded":"2023-12-07T17:16:06.491549+0000","last_fullsized":"2023-12-07T17:16:06.491549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T03:19:50.713717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992994+0000","last_change":"2023-12-07T17:15:09.739107+0000","last_active":"2023-12-07T17:16:06.992994+0000","last_peered":"2023-12-07T17:16:06.992994+0000","last_clean":"2023-12-07T17:16:06.992994+0000","last_became_active":"2023-12-07T17:15:09.738758+0000","last_became_peered":"2023-12-07T17:15:09.738758+0000","last_unstale":"2023-12-07T17:16:06.992994+0000","last_undegraded":"2023-12-07T17:16:06.992994+0000","last_fullsized":"2023-12-07T17:16:06.992994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T05:12:22.803108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140697+0000","last_change":"2023-12-07T17:15:09.740165+0000","last_active":"2023-12-07T17:16:06.140697+0000","last_peered":"2023-12-07T17:16:06.140697+0000","last_clean":"2023-12-07T17:16:06.140697+0000","last_became_active":"2023-12-07T17:15:09.739820+0000","last_became_peered":"2023-12-07T17:15:09.739820+0000","last_unstale":"2023-12-07T17:16:06.140697+0000","last_undegraded":"2023-12-07T17:16:06.140697+0000","last_fullsized":"2023-12-07T17:16:06.140697+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T02:44:16.243037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491302+0000","last_change":"2023-12-07T17:15:09.742176+0000","last_active":"2023-12-07T17:16:06.491302+0000","last_peered":"2023-12-07T17:16:06.491302+0000","last_clean":"2023-12-07T17:16:06.491302+0000","last_became_active":"2023-12-07T17:15:09.738366+0000","last_became_peered":"2023-12-07T17:15:09.738366+0000","last_unstale":"2023-12-07T17:16:06.491302+0000","last_undegraded":"2023-12-07T17:16:06.491302+0000","last_fullsized":"2023-12-07T17:16:06.491302+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:22:17.951815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.960759+0000","last_change":"2023-12-07T17:15:09.742056+0000","last_active":"2023-12-07T17:16:05.960759+0000","last_peered":"2023-12-07T17:16:05.960759+0000","last_clean":"2023-12-07T17:16:05.960759+0000","last_became_active":"2023-12-07T17:15:09.741708+0000","last_became_peered":"2023-12-07T17:15:09.741708+0000","last_unstale":"2023-12-07T17:16:05.960759+0000","last_undegraded":"2023-12-07T17:16:05.960759+0000","last_fullsized":"2023-12-07T17:16:05.960759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:52:12.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992259+0000","last_change":"2023-12-07T17:15:09.735452+0000","last_active":"2023-12-07T17:16:06.992259+0000","last_peered":"2023-12-07T17:16:06.992259+0000","last_clean":"2023-12-07T17:16:06.992259+0000","last_became_active":"2023-12-07T17:15:09.734922+0000","last_became_peered":"2023-12-07T17:15:09.734922+0000","last_unstale":"2023-12-07T17:16:06.992259+0000","last_undegraded":"2023-12-07T17:16:06.992259+0000","last_fullsized":"2023-12-07T17:16:06.992259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T05:07:43.090081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491141+0000","last_change":"2023-12-07T17:15:09.734421+0000","last_active":"2023-12-07T17:16:06.491141+0000","last_peered":"2023-12-07T17:16:06.491141+0000","last_clean":"2023-12-07T17:16:06.491141+0000","last_became_active":"2023-12-07T17:15:09.734134+0000","last_became_peered":"2023-12-07T17:15:09.734134+0000","last_unstale":"2023-12-07T17:16:06.491141+0000","last_undegraded":"2023-12-07T17:16:06.491141+0000","last_fullsized":"2023-12-07T17:16:06.491141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:39.069901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991187+0000","last_change":"2023-12-07T17:15:09.738156+0000","last_active":"2023-12-07T17:16:06.991187+0000","last_peered":"2023-12-07T17:16:06.991187+0000","last_clean":"2023-12-07T17:16:06.991187+0000","last_became_active":"2023-12-07T17:15:09.737738+0000","last_became_peered":"2023-12-07T17:15:09.737738+0000","last_unstale":"2023-12-07T17:16:06.991187+0000","last_undegraded":"2023-12-07T17:16:06.991187+0000","last_fullsized":"2023-12-07T17:16:06.991187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:18:49.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.962500+0000","last_change":"2023-12-07T17:15:09.740051+0000","last_active":"2023-12-07T17:16:05.962500+0000","last_peered":"2023-12-07T17:16:05.962500+0000","last_clean":"2023-12-07T17:16:05.962500+0000","last_became_active":"2023-12-07T17:15:09.739752+0000","last_became_peered":"2023-12-07T17:15:09.739752+0000","last_unstale":"2023-12-07T17:16:05.962500+0000","last_undegraded":"2023-12-07T17:16:05.962500+0000","last_fullsized":"2023-12-07T17:16:05.962500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:35:51.955817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140462+0000","last_change":"2023-12-07T17:15:09.740920+0000","last_active":"2023-12-07T17:16:06.140462+0000","last_peered":"2023-12-07T17:16:06.140462+0000","last_clean":"2023-12-07T17:16:06.140462+0000","last_became_active":"2023-12-07T17:15:09.740628+0000","last_became_peered":"2023-12-07T17:15:09.740628+0000","last_unstale":"2023-12-07T17:16:06.140462+0000","last_undegraded":"2023-12-07T17:16:06.140462+0000","last_fullsized":"2023-12-07T17:16:06.140462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T02:34:32.982853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992067+0000","last_change":"2023-12-07T17:15:09.739301+0000","last_active":"2023-12-07T17:16:06.992067+0000","last_peered":"2023-12-07T17:16:06.992067+0000","last_clean":"2023-12-07T17:16:06.992067+0000","last_became_active":"2023-12-07T17:15:09.738819+0000","last_became_peered":"2023-12-07T17:15:09.738819+0000","last_unstale":"2023-12-07T17:16:06.992067+0000","last_undegraded":"2023-12-07T17:16:06.992067+0000","last_fullsized":"2023-12-07T17:16:06.992067+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:27:01.628578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"70'5","reported_seq":192,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:18:10.478241+0000","last_change":"2023-12-07T17:15:22.723013+0000","last_active":"2023-12-07T17:18:10.478241+0000","last_peered":"2023-12-07T17:18:10.478241+0000","last_clean":"2023-12-07T17:18:10.478241+0000","last_became_active":"2023-12-07T17:15:13.560069+0000","last_became_peered":"2023-12-07T17:15:13.560069+0000","last_unstale":"2023-12-07T17:18:10.478241+0000","last_undegraded":"2023-12-07T17:18:10.478241+0000","last_fullsized":"2023-12-07T17:18:10.478241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:25:18.360736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073499100000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":161,"num_read_kb":156,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991147+0000","last_change":"2023-12-07T17:15:09.738077+0000","last_active":"2023-12-07T17:16:06.991147+0000","last_peered":"2023-12-07T17:16:06.991147+0000","last_clean":"2023-12-07T17:16:06.991147+0000","last_became_active":"2023-12-07T17:15:09.737703+0000","last_became_peered":"2023-12-07T17:15:09.737703+0000","last_unstale":"2023-12-07T17:16:06.991147+0000","last_undegraded":"2023-12-07T17:16:06.991147+0000","last_fullsized":"2023-12-07T17:16:06.991147+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:17:16.761902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"67'2","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.963453+0000","last_change":"2023-12-07T17:15:22.723394+0000","last_active":"2023-12-07T17:16:05.963453+0000","last_peered":"2023-12-07T17:16:05.963453+0000","last_clean":"2023-12-07T17:16:05.963453+0000","last_became_active":"2023-12-07T17:15:13.555197+0000","last_became_peered":"2023-12-07T17:15:13.555197+0000","last_unstale":"2023-12-07T17:16:05.963453+0000","last_undegraded":"2023-12-07T17:16:05.963453+0000","last_fullsized":"2023-12-07T17:16:05.963453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:56:25.588179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682451,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992504+0000","last_change":"2023-12-07T17:15:09.733600+0000","last_active":"2023-12-07T17:16:06.992504+0000","last_peered":"2023-12-07T17:16:06.992504+0000","last_clean":"2023-12-07T17:16:06.992504+0000","last_became_active":"2023-12-07T17:15:09.733223+0000","last_became_peered":"2023-12-07T17:15:09.733223+0000","last_unstale":"2023-12-07T17:16:06.992504+0000","last_undegraded":"2023-12-07T17:16:06.992504+0000","last_fullsized":"2023-12-07T17:16:06.992504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T00:14:33.913419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'188","reported_seq":331,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:08.134222+0000","last_change":"2023-12-07T17:14:32.745381+0000","last_active":"2023-12-07T17:16:08.134222+0000","last_peered":"2023-12-07T17:16:08.134222+0000","last_clean":"2023-12-07T17:16:08.134222+0000","last_became_active":"2023-12-07T17:14:32.744449+0000","last_became_peered":"2023-12-07T17:14:32.744449+0000","last_unstale":"2023-12-07T17:16:08.134222+0000","last_undegraded":"2023-12-07T17:16:08.134222+0000","last_fullsized":"2023-12-07T17:16:08.134222+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:09:56.938364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:09:56.938364+0000","last_clean_scrub_stamp":"2023-12-07T17:09:56.938364+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:45:58.588219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:05.960311+0000","last_change":"2023-12-07T17:15:22.953440+0000","last_active":"2023-12-07T17:16:05.960311+0000","last_peered":"2023-12-07T17:16:05.960311+0000","last_clean":"2023-12-07T17:16:05.960311+0000","last_became_active":"2023-12-07T17:15:13.216515+0000","last_became_peered":"2023-12-07T17:15:13.216515+0000","last_unstale":"2023-12-07T17:16:05.960311+0000","last_undegraded":"2023-12-07T17:16:05.960311+0000","last_fullsized":"2023-12-07T17:16:05.960311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:12.194400+0000","last_clean_scrub_stamp":"2023-12-07T17:15:12.194400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T23:51:41.566664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060667900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992183+0000","last_change":"2023-12-07T17:15:09.743429+0000","last_active":"2023-12-07T17:16:06.992183+0000","last_peered":"2023-12-07T17:16:06.992183+0000","last_clean":"2023-12-07T17:16:06.992183+0000","last_became_active":"2023-12-07T17:15:09.742733+0000","last_became_peered":"2023-12-07T17:15:09.742733+0000","last_unstale":"2023-12-07T17:16:06.992183+0000","last_undegraded":"2023-12-07T17:16:06.992183+0000","last_fullsized":"2023-12-07T17:16:06.992183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:24.179470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992033+0000","last_change":"2023-12-07T17:15:09.744152+0000","last_active":"2023-12-07T17:16:06.992033+0000","last_peered":"2023-12-07T17:16:06.992033+0000","last_clean":"2023-12-07T17:16:06.992033+0000","last_became_active":"2023-12-07T17:15:09.743832+0000","last_became_peered":"2023-12-07T17:15:09.743832+0000","last_unstale":"2023-12-07T17:16:06.992033+0000","last_undegraded":"2023-12-07T17:16:06.992033+0000","last_fullsized":"2023-12-07T17:16:06.992033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T03:38:51.956613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"60'2","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140478+0000","last_change":"2023-12-07T17:15:09.732781+0000","last_active":"2023-12-07T17:16:06.140478+0000","last_peered":"2023-12-07T17:16:06.140478+0000","last_clean":"2023-12-07T17:16:06.140478+0000","last_became_active":"2023-12-07T17:15:09.731586+0000","last_became_peered":"2023-12-07T17:15:09.731586+0000","last_unstale":"2023-12-07T17:16:06.140478+0000","last_undegraded":"2023-12-07T17:16:06.140478+0000","last_fullsized":"2023-12-07T17:16:06.140478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:54.633492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.491549+0000","last_change":"2023-12-07T17:15:09.731572+0000","last_active":"2023-12-07T17:16:06.491549+0000","last_peered":"2023-12-07T17:16:06.491549+0000","last_clean":"2023-12-07T17:16:06.491549+0000","last_became_active":"2023-12-07T17:15:09.731239+0000","last_became_peered":"2023-12-07T17:15:09.731239+0000","last_unstale":"2023-12-07T17:16:06.491549+0000","last_undegraded":"2023-12-07T17:16:06.491549+0000","last_fullsized":"2023-12-07T17:16:06.491549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:02:56.787357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991782+0000","last_change":"2023-12-07T17:15:09.743613+0000","last_active":"2023-12-07T17:16:06.991782+0000","last_peered":"2023-12-07T17:16:06.991782+0000","last_clean":"2023-12-07T17:16:06.991782+0000","last_became_active":"2023-12-07T17:15:09.743141+0000","last_became_peered":"2023-12-07T17:15:09.743141+0000","last_unstale":"2023-12-07T17:16:06.991782+0000","last_undegraded":"2023-12-07T17:16:06.991782+0000","last_fullsized":"2023-12-07T17:16:06.991782+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:16:31.499043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140862+0000","last_change":"2023-12-07T17:15:09.729655+0000","last_active":"2023-12-07T17:16:06.140862+0000","last_peered":"2023-12-07T17:16:06.140862+0000","last_clean":"2023-12-07T17:16:06.140862+0000","last_became_active":"2023-12-07T17:15:09.728138+0000","last_became_peered":"2023-12-07T17:15:09.728138+0000","last_unstale":"2023-12-07T17:16:06.140862+0000","last_undegraded":"2023-12-07T17:16:06.140862+0000","last_fullsized":"2023-12-07T17:16:06.140862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T01:48:51.584987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140548+0000","last_change":"2023-12-07T17:15:09.731496+0000","last_active":"2023-12-07T17:16:06.140548+0000","last_peered":"2023-12-07T17:16:06.140548+0000","last_clean":"2023-12-07T17:16:06.140548+0000","last_became_active":"2023-12-07T17:15:09.731025+0000","last_became_peered":"2023-12-07T17:15:09.731025+0000","last_unstale":"2023-12-07T17:16:06.140548+0000","last_undegraded":"2023-12-07T17:16:06.140548+0000","last_fullsized":"2023-12-07T17:16:06.140548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:35:33.108346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.992317+0000","last_change":"2023-12-07T17:15:09.733583+0000","last_active":"2023-12-07T17:16:06.992317+0000","last_peered":"2023-12-07T17:16:06.992317+0000","last_clean":"2023-12-07T17:16:06.992317+0000","last_became_active":"2023-12-07T17:15:09.733241+0000","last_became_peered":"2023-12-07T17:15:09.733241+0000","last_unstale":"2023-12-07T17:16:06.992317+0000","last_undegraded":"2023-12-07T17:16:06.992317+0000","last_fullsized":"2023-12-07T17:16:06.992317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:13:59.244553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991284+0000","last_change":"2023-12-07T17:15:09.738066+0000","last_active":"2023-12-07T17:16:06.991284+0000","last_peered":"2023-12-07T17:16:06.991284+0000","last_clean":"2023-12-07T17:16:06.991284+0000","last_became_active":"2023-12-07T17:15:09.737689+0000","last_became_peered":"2023-12-07T17:15:09.737689+0000","last_unstale":"2023-12-07T17:16:06.991284+0000","last_undegraded":"2023-12-07T17:16:06.991284+0000","last_fullsized":"2023-12-07T17:16:06.991284+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:23:23.277498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.140561+0000","last_change":"2023-12-07T17:15:09.732745+0000","last_active":"2023-12-07T17:16:06.140561+0000","last_peered":"2023-12-07T17:16:06.140561+0000","last_clean":"2023-12-07T17:16:06.140561+0000","last_became_active":"2023-12-07T17:15:09.732219+0000","last_became_peered":"2023-12-07T17:15:09.732219+0000","last_unstale":"2023-12-07T17:16:06.140561+0000","last_undegraded":"2023-12-07T17:16:06.140561+0000","last_fullsized":"2023-12-07T17:16:06.140561+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T04:14:10.820659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-12-07T17:16:06.991338+0000","last_change":"2023-12-07T17:15:09.731637+0000","last_active":"2023-12-07T17:16:06.991338+0000","last_peered":"2023-12-07T17:16:06.991338+0000","last_clean":"2023-12-07T17:16:06.991338+0000","last_became_active":"2023-12-07T17:15:09.731212+0000","last_became_peered":"2023-12-07T17:15:09.731212+0000","last_unstale":"2023-12-07T17:16:06.991338+0000","last_undegraded":"2023-12-07T17:16:06.991338+0000","last_fullsized":"2023-12-07T17:16:06.991338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T17:15:08.702892+0000","last_clean_scrub_stamp":"2023-12-07T17:15:08.702892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:22:45.616688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":161,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28316,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712740,"statfs":{"total":95990841344,"available":95961845760,"internally_reserved":0,"allocated":1331200,"data_stored":906020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97999999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1819999999999999,"5min":1.409,"15min":1.409},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96299999999999997,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.1819999999999999}]},{"osd":1,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93700000000000006,"15min":0.94699999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2629999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.073},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0089999999999999,"15min":1.0289999999999999},"min":{"1min":0.87,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2989999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":1.2210000000000001}]},{"osd":2,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95499999999999996,"15min":0.96599999999999997},"min":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.361,"5min":1.391,"15min":1.391},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95899999999999996,"15min":0.98499999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.137,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.03}]},{"osd":3,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.98899999999999999,"15min":1.0089999999999999},"min":{"1min":0.78600000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.22,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.042,"5min":1.034,"15min":1.048},"min":{"1min":0.77600000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.417,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.2829999999999999}]},{"osd":4,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88800000000000001,"15min":0.91800000000000004},"min":{"1min":0.66900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.99199999999999999,"5min":1.262,"15min":1.262},"last":1.147}]},{"osd":5,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.57699999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.753},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.58299999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.4610000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.329,"5min":1.526,"15min":1.526},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.121,"15min":1.137},"min":{"1min":0.68200000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5069999999999999,"5min":1.536,"15min":1.536},"last":1.335}]}]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28336,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712720,"statfs":{"total":95990841344,"available":95961825280,"internally_reserved":0,"allocated":1351680,"data_stored":908031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90200000000000002},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3500000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89700000000000002,"15min":0.91300000000000003},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.167,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0289999999999999}]},{"osd":1,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.93100000000000005,"15min":0.97399999999999998},"min":{"1min":0.60399999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.113,"5min":1.704,"15min":1.704},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.878},"min":{"1min":0.58399999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":1.282}]},{"osd":2,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.69099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5169999999999999,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.181},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96899999999999997,"15min":0.998},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.2989999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.94799999999999995,"15min":0.94599999999999995},"min":{"1min":0.69399999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.286,"5min":1.623,"15min":1.623},"last":0.879},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.036,"15min":1.0640000000000001},"min":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4379999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.335}]},{"osd":4,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.875,"15min":0.83899999999999997},"min":{"1min":0.55900000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.373,"5min":1.403,"15min":1.403},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86499999999999999,"15min":0.80300000000000005},"min":{"1min":0.63800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.494,"15min":1.494},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92800000000000005,"15min":0.89400000000000002},"min":{"1min":0.68400000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.92900000000000005},"min":{"1min":0.64700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.343,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.069,"15min":1.097},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3049999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.079,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.591,"5min":1.772,"15min":1.772},"last":1.387}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713308,"statfs":{"total":95990841344,"available":95962427392,"internally_reserved":0,"allocated":745472,"data_stored":316724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.60999999999999999},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.30199999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.67700000000000005},"min":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.34100000000000003},"max":{"1min":1.2390000000000001,"5min":1.429,"15min":1.429},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76800000000000002,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.40000000000000002},"max":{"1min":1.1599999999999999,"5min":1.177,"15min":1.177},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.85099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0329999999999999,"5min":1.329,"15min":1.329},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Dec 7 17:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.876,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5089999999999999,"5min":1.671,"15min":1.671},"last":0.78700000000000003}]},{"osd":4,"last update":"Thu Dec 7 17:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0369999999999999,"15min":0.97399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4430000000000001,"5min":1.54,"15min":1.54},"last":1.077},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.28,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Dec 7 17:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0249999999999999,"15min":1.02},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.169,"5min":1.716,"15min":1.716},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.155,"15min":1.1399999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4610000000000001,"5min":1.736,"15min":1.736},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1559999999999999,"15min":1.1910000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.337,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.2370000000000001,"15min":1.292},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.248,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.3620000000000001}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.246,"15min":1.2589999999999999},"min":{"1min":0.88,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.5880000000000001,"5min":1.639,"15min":1.639},"last":1.264},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.23,"15min":1.258},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.425,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.115}]}]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28328,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712728,"statfs":{"total":95990841344,"available":95961833472,"internally_reserved":0,"allocated":1343488,"data_stored":907271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 17:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.751,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.36899999999999999},"max":{"1min":0.91800000000000004,"5min":2.085,"15min":2.085},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.68600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.34100000000000003},"max":{"1min":1.236,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Dec 7 17:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83199999999999996,"15min":0.70699999999999996},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.39900000000000002},"max":{"1min":1.1060000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.71999999999999997},"min":{"1min":0.50600000000000001,"5min":0.42399999999999999,"15min":0.41599999999999998},"max":{"1min":1.079,"5min":1.383,"15min":1.383},"last":0.84199999999999997}]},{"osd":3,"last update":"Thu Dec 7 17:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.93600000000000005,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0449999999999999,"5min":3.25,"15min":3.25},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":0.98799999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1279999999999999,"5min":1.708,"15min":1.708},"last":1.246},{"interface":"front","average":{"1min":0.91400000000000003,"5min":1.0629999999999999,"15min":1.032},"min":{"1min":0.626,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.204,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.085,"15min":1.101},"min":{"1min":0.60499999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2490000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.1120000000000001,"15min":1.093},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.327,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Dec 7 17:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2070000000000001},"min":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3080000000000001,"5min":2.786,"15min":2.786},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1970000000000001,"15min":1.2250000000000001},"min":{"1min":0.88600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.496,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Dec 7 17:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.268,"15min":1.302},"min":{"1min":0.84699999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.601,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.224,"15min":1.264},"min":{"1min":0.71199999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.7070000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.135}]}]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713316,"statfs":{"total":95990841344,"available":95962435584,"internally_reserved":0,"allocated":737280,"data_stored":316280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.751,"15min":0.66000000000000003},"min":{"1min":0.372,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.125,"5min":3.02,"15min":3.02},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.84899999999999998,"15min":0.72799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2729999999999999,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.81299999999999994,"15min":0.68899999999999995},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.374},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.628},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.88700000000000001,"15min":0.75600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.871,"5min":1.454,"15min":1.454},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.90600000000000003,"15min":0.85999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":2.101,"15min":2.101},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.86899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2869999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Dec 7 17:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.96599999999999997,"15min":0.93799999999999994},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.131,"15min":1.075},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3089999999999999,"5min":1.948,"15min":1.948},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Dec 7 17:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.1240000000000001,"15min":1.1399999999999999},"min":{"1min":0.65500000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4650000000000001,"5min":1.673,"15min":1.673},"last":1.234},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0249999999999999,"15min":1.0409999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.381,"5min":1.784,"15min":1.784},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Dec 7 17:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.196,"15min":1.224},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.367,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.206,"15min":1.2050000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.764,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.85999999999999999}]},{"osd":7,"last update":"Thu Dec 7 17:17:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.129,"15min":1.161},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4239999999999999,"5min":1.899,"15min":1.899},"last":1.27},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.3180000000000001,"15min":1.3799999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8520000000000001,"5min":2.133,"15min":2.133},"last":1.113}]}]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713308,"statfs":{"total":95990841344,"available":95962427392,"internally_reserved":0,"allocated":745472,"data_stored":316830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.75900000000000001,"15min":0.68600000000000005},"min":{"1min":0.40400000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.252,"5min":1.306,"15min":1.306},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.014,"5min":1.25,"15min":1.25},"last":0.62}]},{"osd":1,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.81899999999999995,"15min":0.79400000000000004},"min":{"1min":0.42799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.254,"5min":1.262,"15min":1.262},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0840000000000001,"5min":1.228,"15min":1.625},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Dec 7 17:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.88500000000000001,"15min":0.86799999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.167,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.873,"15min":0.84599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2110000000000001,"5min":1.647,"15min":1.647},"last":0.83799999999999997}]},{"osd":4,"last update":"Thu Dec 7 17:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0409999999999999,"15min":0.97899999999999998},"min":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4890000000000001,"5min":1.522,"15min":1.522},"last":0.88},{"interface":"front","average":{"1min":1.022,"5min":1.091,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.353,"5min":1.829,"15min":1.829},"last":0.96999999999999997}]},{"osd":5,"last update":"Thu Dec 7 17:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1100000000000001,"15min":1.0780000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5569999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.131},{"interface":"front","average":{"1min":1.016,"5min":1.103,"15min":1.0860000000000001},"min":{"1min":0.68600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3460000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.044}]},{"osd":6,"last update":"Thu Dec 7 17:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1830000000000001,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.655,"5min":1.927,"15min":1.927},"last":1.079},{"interface":"front","average":{"1min":1.093,"5min":1.1619999999999999,"15min":1.1719999999999999},"min":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5049999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Thu Dec 7 17:17:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.216,"15min":1.244},"min":{"1min":0.79800000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5840000000000001,"5min":1.875,"15min":1.875},"last":1.238},{"interface":"front","average":{"1min":1.073,"5min":1.151,"15min":1.157},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.544,"5min":1.583,"15min":1.583},"last":1.1020000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018959,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713304,"statfs":{"total":95990841344,"available":95962423296,"internally_reserved":0,"allocated":749568,"data_stored":316427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.86199999999999999},"min":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"max":{"1min":1.097,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.89900000000000002,"15min":0.92500000000000004},"min":{"1min":0.50600000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.91400000000000003,"15min":0.88800000000000001},"min":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.204,"5min":1.425,"15min":1.425},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93100000000000005,"15min":0.93999999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.016}]},{"osd":2,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93400000000000005,"15min":0.94999999999999996},"min":{"1min":0.66700000000000004,"5min":0.63,"15min":0.63},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3049999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.97599999999999998}]},{"osd":3,"last update":"Thu Dec 7 17:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96599999999999997,"15min":0.95999999999999996},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0029999999999999,"15min":1.0009999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3580000000000001,"5min":1.385,"15min":1.385},"last":0.93200000000000005}]},{"osd":5,"last update":"Thu Dec 7 17:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85899999999999999,"15min":0.77300000000000002},"min":{"1min":0.50700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.98399999999999999,"15min":0.90600000000000003},"min":{"1min":0.56100000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.383,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.429}]},{"osd":6,"last update":"Thu Dec 7 17:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.296,"5min":1.581,"15min":1.581},"last":1.3660000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94199999999999995,"15min":0.89600000000000002},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4139999999999999,"5min":1.448,"15min":1.448},"last":1.179}]},{"osd":7,"last update":"Thu Dec 7 17:17:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.052},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4419999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.008,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4330000000000001,"5min":1.536,"15min":1.536},"last":1.1559999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713316,"statfs":{"total":95990841344,"available":95962435584,"internally_reserved":0,"allocated":737280,"data_stored":315910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.91800000000000004,"15min":0.89300000000000002},"min":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.125,"5min":1.464,"15min":1.464},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.91400000000000003},"min":{"1min":0.62,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1639999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.92100000000000004},"min":{"1min":0.73499999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1579999999999999,"5min":1.286,"15min":1.286},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.89100000000000001,"15min":0.86499999999999999},"min":{"1min":0.56100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.155,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97499999999999998,"15min":0.89500000000000002},"min":{"1min":0.81100000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0009999999999999,"15min":0.96199999999999997},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97199999999999998,"15min":0.94199999999999995},"min":{"1min":0.67900000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0009999999999999},"min":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.034}]},{"osd":4,"last update":"Thu Dec 7 17:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.82899999999999996,"15min":0.76800000000000002},"min":{"1min":0.58199999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.179,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.91200000000000003,"15min":0.86099999999999999},"min":{"1min":0.54200000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89200000000000002,"15min":0.85999999999999999},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1859999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.139},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":0.97599999999999998},"min":{"1min":0.68400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.294,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.113}]},{"osd":7,"last update":"Thu Dec 7 17:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2569999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.091},{"interface":"front","average":{"1min":1,"5min":1.0860000000000001,"15min":1.1180000000000001},"min":{"1min":0.72599999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2849999999999999,"5min":1.51,"15min":1.51},"last":1.2250000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T17:18:17.187 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-07T17:18:17.187 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-07T17:18:17.187 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-07T17:18:17.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:18:17.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:18.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:18 smithi184 bash[31336]: cluster 2023-12-07T17:18:16.257415+0000 mgr.y (mgr.24626) 118 : cluster 0 pgmap v72: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:18.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:18 smithi184 bash[31336]: audit 2023-12-07T17:18:16.339203+0000 mgr.y (mgr.24626) 119 : audit 0 from='client.24818 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:18.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:18 smithi184 bash[31336]: audit 2023-12-07T17:18:17.097680+0000 mgr.y (mgr.24626) 120 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:18.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[25056]: cluster 2023-12-07T17:18:16.257415+0000 mgr.y (mgr.24626) 118 : cluster 0 pgmap v72: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:18.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[25056]: audit 2023-12-07T17:18:16.339203+0000 mgr.y (mgr.24626) 119 : audit 0 from='client.24818 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:18.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[25056]: audit 2023-12-07T17:18:17.097680+0000 mgr.y (mgr.24626) 120 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:18.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[36207]: cluster 2023-12-07T17:18:16.257415+0000 mgr.y (mgr.24626) 118 : cluster 0 pgmap v72: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:18.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[36207]: audit 2023-12-07T17:18:16.339203+0000 mgr.y (mgr.24626) 119 : audit 0 from='client.24818 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T17:18:18.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:18 smithi064 bash[36207]: audit 2023-12-07T17:18:17.097680+0000 mgr.y (mgr.24626) 120 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:20.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:20 smithi184 bash[31336]: cluster 2023-12-07T17:18:18.258276+0000 mgr.y (mgr.24626) 121 : cluster 0 pgmap v73: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:20 smithi064 bash[25056]: cluster 2023-12-07T17:18:18.258276+0000 mgr.y (mgr.24626) 121 : cluster 0 pgmap v73: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:20.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:20 smithi064 bash[36207]: cluster 2023-12-07T17:18:18.258276+0000 mgr.y (mgr.24626) 121 : cluster 0 pgmap v73: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:21.962 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:22.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:22 smithi064 bash[36207]: cluster 2023-12-07T17:18:20.258889+0000 mgr.y (mgr.24626) 122 : cluster 0 pgmap v74: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:22.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:22 smithi064 bash[36207]: audit 2023-12-07T17:18:21.297594+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:22 smithi064 bash[25056]: cluster 2023-12-07T17:18:20.258889+0000 mgr.y (mgr.24626) 122 : cluster 0 pgmap v74: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:22 smithi064 bash[25056]: audit 2023-12-07T17:18:21.297594+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:22.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:22 smithi184 bash[31336]: cluster 2023-12-07T17:18:20.258889+0000 mgr.y (mgr.24626) 122 : cluster 0 pgmap v74: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:22.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:22 smithi184 bash[31336]: audit 2023-12-07T17:18:21.297594+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:24.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:24 smithi064 bash[25056]: cluster 2023-12-07T17:18:22.259871+0000 mgr.y (mgr.24626) 123 : cluster 0 pgmap v75: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:24.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:24 smithi064 bash[36207]: cluster 2023-12-07T17:18:22.259871+0000 mgr.y (mgr.24626) 123 : cluster 0 pgmap v75: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:24.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:24 smithi184 bash[31336]: cluster 2023-12-07T17:18:22.259871+0000 mgr.y (mgr.24626) 123 : cluster 0 pgmap v75: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:25.233 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:18:25.234 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:18:25.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:26.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:26 smithi064 bash[25056]: cluster 2023-12-07T17:18:24.260567+0000 mgr.y (mgr.24626) 124 : cluster 0 pgmap v76: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:26.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:26 smithi064 bash[25056]: audit 2023-12-07T17:18:25.231093+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.64:0/3742817390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:26.139 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:26 smithi064 bash[36207]: cluster 2023-12-07T17:18:24.260567+0000 mgr.y (mgr.24626) 124 : cluster 0 pgmap v76: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:26.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:26 smithi064 bash[36207]: audit 2023-12-07T17:18:25.231093+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.64:0/3742817390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:26.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:26 smithi184 bash[31336]: cluster 2023-12-07T17:18:24.260567+0000 mgr.y (mgr.24626) 124 : cluster 0 pgmap v76: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:26.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:26 smithi184 bash[31336]: audit 2023-12-07T17:18:25.231093+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.64:0/3742817390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:27.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:28.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:28 smithi184 bash[31336]: cluster 2023-12-07T17:18:26.261185+0000 mgr.y (mgr.24626) 125 : cluster 0 pgmap v77: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:28.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:28 smithi184 bash[31336]: audit 2023-12-07T17:18:27.106835+0000 mgr.y (mgr.24626) 126 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:28.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:28 smithi064 bash[36207]: cluster 2023-12-07T17:18:26.261185+0000 mgr.y (mgr.24626) 125 : cluster 0 pgmap v77: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:28.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:28 smithi064 bash[36207]: audit 2023-12-07T17:18:27.106835+0000 mgr.y (mgr.24626) 126 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:28.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:28 smithi064 bash[25056]: cluster 2023-12-07T17:18:26.261185+0000 mgr.y (mgr.24626) 125 : cluster 0 pgmap v77: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:28.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:28 smithi064 bash[25056]: audit 2023-12-07T17:18:27.106835+0000 mgr.y (mgr.24626) 126 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:29.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:18:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:30 smithi184 bash[31336]: cluster 2023-12-07T17:18:28.262031+0000 mgr.y (mgr.24626) 127 : cluster 0 pgmap v78: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:30.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:30 smithi064 bash[25056]: cluster 2023-12-07T17:18:28.262031+0000 mgr.y (mgr.24626) 127 : cluster 0 pgmap v78: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:30.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:30 smithi064 bash[36207]: cluster 2023-12-07T17:18:28.262031+0000 mgr.y (mgr.24626) 127 : cluster 0 pgmap v78: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:32.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:32 smithi184 bash[31336]: cluster 2023-12-07T17:18:30.262655+0000 mgr.y (mgr.24626) 128 : cluster 0 pgmap v79: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:32.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:32 smithi064 bash[25056]: cluster 2023-12-07T17:18:30.262655+0000 mgr.y (mgr.24626) 128 : cluster 0 pgmap v79: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:32.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:32 smithi064 bash[36207]: cluster 2023-12-07T17:18:30.262655+0000 mgr.y (mgr.24626) 128 : cluster 0 pgmap v79: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:33.877 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:34.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:34 smithi064 bash[25056]: cluster 2023-12-07T17:18:32.263655+0000 mgr.y (mgr.24626) 129 : cluster 0 pgmap v80: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:34.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:34 smithi064 bash[36207]: cluster 2023-12-07T17:18:32.263655+0000 mgr.y (mgr.24626) 129 : cluster 0 pgmap v80: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:34.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:34 smithi184 bash[31336]: cluster 2023-12-07T17:18:32.263655+0000 mgr.y (mgr.24626) 129 : cluster 0 pgmap v80: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:35.950 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:35 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:36.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:36 smithi064 bash[25056]: cluster 2023-12-07T17:18:34.264360+0000 mgr.y (mgr.24626) 130 : cluster 0 pgmap v81: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:36.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:36 smithi064 bash[36207]: cluster 2023-12-07T17:18:34.264360+0000 mgr.y (mgr.24626) 130 : cluster 0 pgmap v81: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:36.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:36 smithi184 bash[31336]: cluster 2023-12-07T17:18:34.264360+0000 mgr.y (mgr.24626) 130 : cluster 0 pgmap v81: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:37.384 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:18:37.384 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:18:37.437 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:37 smithi064 bash[36207]: audit 2023-12-07T17:18:36.315705+0000 mon.c (mon.2) 75 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:37.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:37 smithi064 bash[25056]: audit 2023-12-07T17:18:36.315705+0000 mon.c (mon.2) 75 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:37.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:37 smithi184 bash[31336]: audit 2023-12-07T17:18:36.315705+0000 mon.c (mon.2) 75 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:37.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:38 smithi184 bash[31336]: cluster 2023-12-07T17:18:36.264999+0000 mgr.y (mgr.24626) 131 : cluster 0 pgmap v82: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:38 smithi184 bash[31336]: audit 2023-12-07T17:18:37.114868+0000 mgr.y (mgr.24626) 132 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:38 smithi184 bash[31336]: audit 2023-12-07T17:18:37.381467+0000 mon.c (mon.2) 76 : audit 0 from='client.? 172.21.15.64:0/158426348' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[25056]: cluster 2023-12-07T17:18:36.264999+0000 mgr.y (mgr.24626) 131 : cluster 0 pgmap v82: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[25056]: audit 2023-12-07T17:18:37.114868+0000 mgr.y (mgr.24626) 132 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[25056]: audit 2023-12-07T17:18:37.381467+0000 mon.c (mon.2) 76 : audit 0 from='client.? 172.21.15.64:0/158426348' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[36207]: cluster 2023-12-07T17:18:36.264999+0000 mgr.y (mgr.24626) 131 : cluster 0 pgmap v82: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[36207]: audit 2023-12-07T17:18:37.114868+0000 mgr.y (mgr.24626) 132 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:38.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:38 smithi064 bash[36207]: audit 2023-12-07T17:18:37.381467+0000 mon.c (mon.2) 76 : audit 0 from='client.? 172.21.15.64:0/158426348' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:40.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:40 smithi184 bash[31336]: cluster 2023-12-07T17:18:38.265950+0000 mgr.y (mgr.24626) 133 : cluster 0 pgmap v83: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:40.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:40 smithi064 bash[36207]: cluster 2023-12-07T17:18:38.265950+0000 mgr.y (mgr.24626) 133 : cluster 0 pgmap v83: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:40.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:40 smithi064 bash[25056]: cluster 2023-12-07T17:18:38.265950+0000 mgr.y (mgr.24626) 133 : cluster 0 pgmap v83: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:41.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:18:41.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:41 smithi184 bash[31336]: audit 2023-12-07T17:18:40.333730+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:18:41.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:41 smithi184 bash[31336]: audit 2023-12-07T17:18:41.060452+0000 mon.c (mon.2) 78 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:18:41.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:41 smithi184 bash[31336]: audit 2023-12-07T17:18:41.062264+0000 mon.c (mon.2) 79 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:18:41.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:41 smithi184 bash[31336]: audit 2023-12-07T17:18:41.074684+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:18:41.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[25056]: audit 2023-12-07T17:18:40.333730+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:18:41.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[25056]: audit 2023-12-07T17:18:41.060452+0000 mon.c (mon.2) 78 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:18:41.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[25056]: audit 2023-12-07T17:18:41.062264+0000 mon.c (mon.2) 79 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:18:41.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[25056]: audit 2023-12-07T17:18:41.074684+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:18:41.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[36207]: audit 2023-12-07T17:18:40.333730+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:18:41.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[36207]: audit 2023-12-07T17:18:41.060452+0000 mon.c (mon.2) 78 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:18:41.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[36207]: audit 2023-12-07T17:18:41.062264+0000 mon.c (mon.2) 79 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:18:41.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:41 smithi064 bash[36207]: audit 2023-12-07T17:18:41.074684+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:18:42.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:42 smithi184 bash[31336]: cluster 2023-12-07T17:18:40.266572+0000 mgr.y (mgr.24626) 134 : cluster 0 pgmap v84: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:42.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:42 smithi184 bash[31336]: cluster 2023-12-07T17:18:41.065896+0000 mgr.y (mgr.24626) 135 : cluster 0 pgmap v85: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 0 op/s 2023-12-07T17:18:42.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:42 smithi184 bash[31336]: cluster 2023-12-07T17:18:41.066240+0000 mgr.y (mgr.24626) 136 : cluster 0 pgmap v86: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 581 B/s rd, 0 op/s 2023-12-07T17:18:42.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[25056]: cluster 2023-12-07T17:18:40.266572+0000 mgr.y (mgr.24626) 134 : cluster 0 pgmap v84: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:42.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[25056]: cluster 2023-12-07T17:18:41.065896+0000 mgr.y (mgr.24626) 135 : cluster 0 pgmap v85: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 0 op/s 2023-12-07T17:18:42.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[25056]: cluster 2023-12-07T17:18:41.066240+0000 mgr.y (mgr.24626) 136 : cluster 0 pgmap v86: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 581 B/s rd, 0 op/s 2023-12-07T17:18:42.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[36207]: cluster 2023-12-07T17:18:40.266572+0000 mgr.y (mgr.24626) 134 : cluster 0 pgmap v84: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:42.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[36207]: cluster 2023-12-07T17:18:41.065896+0000 mgr.y (mgr.24626) 135 : cluster 0 pgmap v85: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 0 op/s 2023-12-07T17:18:42.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:42 smithi064 bash[36207]: cluster 2023-12-07T17:18:41.066240+0000 mgr.y (mgr.24626) 136 : cluster 0 pgmap v86: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 581 B/s rd, 0 op/s 2023-12-07T17:18:44.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:44 smithi184 bash[31336]: cluster 2023-12-07T17:18:43.067078+0000 mgr.y (mgr.24626) 137 : cluster 0 pgmap v87: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:44.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:44 smithi064 bash[25056]: cluster 2023-12-07T17:18:43.067078+0000 mgr.y (mgr.24626) 137 : cluster 0 pgmap v87: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:44.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:44 smithi064 bash[36207]: cluster 2023-12-07T17:18:43.067078+0000 mgr.y (mgr.24626) 137 : cluster 0 pgmap v87: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:45.962 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:45.990 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:46.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:46 smithi064 bash[25056]: cluster 2023-12-07T17:18:45.067738+0000 mgr.y (mgr.24626) 138 : cluster 0 pgmap v88: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:46.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:46 smithi064 bash[36207]: cluster 2023-12-07T17:18:45.067738+0000 mgr.y (mgr.24626) 138 : cluster 0 pgmap v88: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:46.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:46 smithi184 bash[31336]: cluster 2023-12-07T17:18:45.067738+0000 mgr.y (mgr.24626) 138 : cluster 0 pgmap v88: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:47.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:48.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:48 smithi064 bash[25056]: cluster 2023-12-07T17:18:47.068681+0000 mgr.y (mgr.24626) 139 : cluster 0 pgmap v89: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:48.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:48 smithi064 bash[25056]: audit 2023-12-07T17:18:47.124641+0000 mgr.y (mgr.24626) 140 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:48.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:48 smithi064 bash[36207]: cluster 2023-12-07T17:18:47.068681+0000 mgr.y (mgr.24626) 139 : cluster 0 pgmap v89: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:48.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:48 smithi064 bash[36207]: audit 2023-12-07T17:18:47.124641+0000 mgr.y (mgr.24626) 140 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:48.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:48 smithi184 bash[31336]: cluster 2023-12-07T17:18:47.068681+0000 mgr.y (mgr.24626) 139 : cluster 0 pgmap v89: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:48.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:48 smithi184 bash[31336]: audit 2023-12-07T17:18:47.124641+0000 mgr.y (mgr.24626) 140 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:49.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:18:49.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:18:50.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:50 smithi184 bash[31336]: cluster 2023-12-07T17:18:49.069471+0000 mgr.y (mgr.24626) 141 : cluster 0 pgmap v90: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:50.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:50 smithi184 bash[31336]: audit 2023-12-07T17:18:49.386164+0000 mon.a (mon.0) 817 : audit 0 from='client.? 172.21.15.64:0/772058166' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:50.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:50 smithi064 bash[25056]: cluster 2023-12-07T17:18:49.069471+0000 mgr.y (mgr.24626) 141 : cluster 0 pgmap v90: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:50.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:50 smithi064 bash[25056]: audit 2023-12-07T17:18:49.386164+0000 mon.a (mon.0) 817 : audit 0 from='client.? 172.21.15.64:0/772058166' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:50.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:50 smithi064 bash[36207]: cluster 2023-12-07T17:18:49.069471+0000 mgr.y (mgr.24626) 141 : cluster 0 pgmap v90: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:18:50.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:50 smithi064 bash[36207]: audit 2023-12-07T17:18:49.386164+0000 mon.a (mon.0) 817 : audit 0 from='client.? 172.21.15.64:0/772058166' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:18:52.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:51 smithi184 bash[31336]: cluster 2023-12-07T17:18:51.070150+0000 mgr.y (mgr.24626) 142 : cluster 0 pgmap v91: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:18:52.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:51 smithi184 bash[31336]: audit 2023-12-07T17:18:51.333879+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:51 smithi064 bash[36207]: cluster 2023-12-07T17:18:51.070150+0000 mgr.y (mgr.24626) 142 : cluster 0 pgmap v91: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:18:52.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:51 smithi064 bash[36207]: audit 2023-12-07T17:18:51.333879+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:52.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:51 smithi064 bash[25056]: cluster 2023-12-07T17:18:51.070150+0000 mgr.y (mgr.24626) 142 : cluster 0 pgmap v91: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:18:52.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:51 smithi064 bash[25056]: audit 2023-12-07T17:18:51.333879+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:18:53.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:18:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:54 smithi184 bash[31336]: cluster 2023-12-07T17:18:53.071028+0000 mgr.y (mgr.24626) 143 : cluster 0 pgmap v92: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:54.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:54 smithi064 bash[25056]: cluster 2023-12-07T17:18:53.071028+0000 mgr.y (mgr.24626) 143 : cluster 0 pgmap v92: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:54.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:54 smithi064 bash[36207]: cluster 2023-12-07T17:18:53.071028+0000 mgr.y (mgr.24626) 143 : cluster 0 pgmap v92: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:56.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:18:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:18:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:18:56.569 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:56 smithi184 bash[31336]: cluster 2023-12-07T17:18:55.071709+0000 mgr.y (mgr.24626) 144 : cluster 0 pgmap v93: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:56.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:56 smithi064 bash[25056]: cluster 2023-12-07T17:18:55.071709+0000 mgr.y (mgr.24626) 144 : cluster 0 pgmap v93: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:56.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:56 smithi064 bash[36207]: cluster 2023-12-07T17:18:55.071709+0000 mgr.y (mgr.24626) 144 : cluster 0 pgmap v93: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:18:57.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:18:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:18:57.957 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:18:58.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:58 smithi064 bash[25056]: cluster 2023-12-07T17:18:57.072503+0000 mgr.y (mgr.24626) 145 : cluster 0 pgmap v94: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:58.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:18:58 smithi064 bash[25056]: audit 2023-12-07T17:18:57.133305+0000 mgr.y (mgr.24626) 146 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:58.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:58 smithi064 bash[36207]: cluster 2023-12-07T17:18:57.072503+0000 mgr.y (mgr.24626) 145 : cluster 0 pgmap v94: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:58.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:18:58 smithi064 bash[36207]: audit 2023-12-07T17:18:57.133305+0000 mgr.y (mgr.24626) 146 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:18:58.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:58 smithi184 bash[31336]: cluster 2023-12-07T17:18:57.072503+0000 mgr.y (mgr.24626) 145 : cluster 0 pgmap v94: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:18:58.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:18:58 smithi184 bash[31336]: audit 2023-12-07T17:18:57.133305+0000 mgr.y (mgr.24626) 146 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:00.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:00 smithi064 bash[36207]: cluster 2023-12-07T17:18:59.073026+0000 mgr.y (mgr.24626) 147 : cluster 0 pgmap v95: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:00.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:00 smithi064 bash[25056]: cluster 2023-12-07T17:18:59.073026+0000 mgr.y (mgr.24626) 147 : cluster 0 pgmap v95: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:00.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:00 smithi184 bash[31336]: cluster 2023-12-07T17:18:59.073026+0000 mgr.y (mgr.24626) 147 : cluster 0 pgmap v95: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:01.422 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:19:01.422 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:19:02.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:01 smithi184 bash[31336]: cluster 2023-12-07T17:19:01.073407+0000 mgr.y (mgr.24626) 148 : cluster 0 pgmap v96: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:02.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:01 smithi184 bash[31336]: audit 2023-12-07T17:19:01.419745+0000 mon.a (mon.0) 818 : audit 0 from='client.? 172.21.15.64:0/3046986942' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:02.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:01 smithi064 bash[25056]: cluster 2023-12-07T17:19:01.073407+0000 mgr.y (mgr.24626) 148 : cluster 0 pgmap v96: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:02.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:01 smithi064 bash[25056]: audit 2023-12-07T17:19:01.419745+0000 mon.a (mon.0) 818 : audit 0 from='client.? 172.21.15.64:0/3046986942' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:02.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:01 smithi064 bash[36207]: cluster 2023-12-07T17:19:01.073407+0000 mgr.y (mgr.24626) 148 : cluster 0 pgmap v96: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:02.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:01 smithi064 bash[36207]: audit 2023-12-07T17:19:01.419745+0000 mon.a (mon.0) 818 : audit 0 from='client.? 172.21.15.64:0/3046986942' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:04.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:04 smithi184 bash[31336]: cluster 2023-12-07T17:19:03.074238+0000 mgr.y (mgr.24626) 149 : cluster 0 pgmap v97: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:04.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:04 smithi064 bash[36207]: cluster 2023-12-07T17:19:03.074238+0000 mgr.y (mgr.24626) 149 : cluster 0 pgmap v97: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:04.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:04 smithi064 bash[25056]: cluster 2023-12-07T17:19:03.074238+0000 mgr.y (mgr.24626) 149 : cluster 0 pgmap v97: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:05.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:19:06.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:06.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:06 smithi184 bash[31336]: cluster 2023-12-07T17:19:05.074828+0000 mgr.y (mgr.24626) 150 : cluster 0 pgmap v98: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:06.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:06 smithi064 bash[25056]: cluster 2023-12-07T17:19:05.074828+0000 mgr.y (mgr.24626) 150 : cluster 0 pgmap v98: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:06.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:06 smithi064 bash[36207]: cluster 2023-12-07T17:19:05.074828+0000 mgr.y (mgr.24626) 150 : cluster 0 pgmap v98: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:07.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:07.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:07 smithi184 bash[31336]: audit 2023-12-07T17:19:06.351647+0000 mon.c (mon.2) 81 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:07.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:07 smithi064 bash[25056]: audit 2023-12-07T17:19:06.351647+0000 mon.c (mon.2) 81 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:07.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:07 smithi064 bash[36207]: audit 2023-12-07T17:19:06.351647+0000 mon.c (mon.2) 81 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:08.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:08 smithi184 bash[31336]: cluster 2023-12-07T17:19:07.075768+0000 mgr.y (mgr.24626) 151 : cluster 0 pgmap v99: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:08.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:08 smithi184 bash[31336]: audit 2023-12-07T17:19:07.142860+0000 mgr.y (mgr.24626) 152 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:08.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:08 smithi064 bash[25056]: cluster 2023-12-07T17:19:07.075768+0000 mgr.y (mgr.24626) 151 : cluster 0 pgmap v99: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:08.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:08 smithi064 bash[25056]: audit 2023-12-07T17:19:07.142860+0000 mgr.y (mgr.24626) 152 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:08.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:08 smithi064 bash[36207]: cluster 2023-12-07T17:19:07.075768+0000 mgr.y (mgr.24626) 151 : cluster 0 pgmap v99: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:08.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:08 smithi064 bash[36207]: audit 2023-12-07T17:19:07.142860+0000 mgr.y (mgr.24626) 152 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:09.924 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:19:10.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:10 smithi064 bash[25056]: cluster 2023-12-07T17:19:09.076560+0000 mgr.y (mgr.24626) 153 : cluster 0 pgmap v100: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:10.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:10 smithi064 bash[36207]: cluster 2023-12-07T17:19:09.076560+0000 mgr.y (mgr.24626) 153 : cluster 0 pgmap v100: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:10.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:10 smithi184 bash[31336]: cluster 2023-12-07T17:19:09.076560+0000 mgr.y (mgr.24626) 153 : cluster 0 pgmap v100: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:12.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:11 smithi184 bash[31336]: cluster 2023-12-07T17:19:11.077262+0000 mgr.y (mgr.24626) 154 : cluster 0 pgmap v101: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:12.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:11 smithi064 bash[25056]: cluster 2023-12-07T17:19:11.077262+0000 mgr.y (mgr.24626) 154 : cluster 0 pgmap v101: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:12.104 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:11 smithi064 bash[36207]: cluster 2023-12-07T17:19:11.077262+0000 mgr.y (mgr.24626) 154 : cluster 0 pgmap v101: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:13.584 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:19:13.584 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:19:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:14 smithi184 bash[31336]: cluster 2023-12-07T17:19:13.078128+0000 mgr.y (mgr.24626) 155 : cluster 0 pgmap v102: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:14 smithi184 bash[31336]: audit 2023-12-07T17:19:13.581547+0000 mon.a (mon.0) 819 : audit 0 from='client.? 172.21.15.64:0/1349233554' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:14 smithi064 bash[25056]: cluster 2023-12-07T17:19:13.078128+0000 mgr.y (mgr.24626) 155 : cluster 0 pgmap v102: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:14.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:14 smithi064 bash[25056]: audit 2023-12-07T17:19:13.581547+0000 mon.a (mon.0) 819 : audit 0 from='client.? 172.21.15.64:0/1349233554' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:14 smithi064 bash[36207]: cluster 2023-12-07T17:19:13.078128+0000 mgr.y (mgr.24626) 155 : cluster 0 pgmap v102: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:14 smithi064 bash[36207]: audit 2023-12-07T17:19:13.581547+0000 mon.a (mon.0) 819 : audit 0 from='client.? 172.21.15.64:0/1349233554' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:16.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:16.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:16 smithi184 bash[31336]: cluster 2023-12-07T17:19:15.078773+0000 mgr.y (mgr.24626) 156 : cluster 0 pgmap v103: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:16.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:16 smithi064 bash[25056]: cluster 2023-12-07T17:19:15.078773+0000 mgr.y (mgr.24626) 156 : cluster 0 pgmap v103: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:16.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:16 smithi064 bash[36207]: cluster 2023-12-07T17:19:15.078773+0000 mgr.y (mgr.24626) 156 : cluster 0 pgmap v103: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:17.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:19:17.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:18.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:18 smithi184 bash[31336]: cluster 2023-12-07T17:19:17.079636+0000 mgr.y (mgr.24626) 157 : cluster 0 pgmap v104: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:18.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:18 smithi184 bash[31336]: audit 2023-12-07T17:19:17.150871+0000 mgr.y (mgr.24626) 158 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:18.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:18 smithi064 bash[36207]: cluster 2023-12-07T17:19:17.079636+0000 mgr.y (mgr.24626) 157 : cluster 0 pgmap v104: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:18.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:18 smithi064 bash[36207]: audit 2023-12-07T17:19:17.150871+0000 mgr.y (mgr.24626) 158 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:18.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:18 smithi064 bash[25056]: cluster 2023-12-07T17:19:17.079636+0000 mgr.y (mgr.24626) 157 : cluster 0 pgmap v104: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:18.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:18 smithi064 bash[25056]: audit 2023-12-07T17:19:17.150871+0000 mgr.y (mgr.24626) 158 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:20.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:20 smithi184 bash[31336]: cluster 2023-12-07T17:19:19.080247+0000 mgr.y (mgr.24626) 159 : cluster 0 pgmap v105: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:20.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:20 smithi064 bash[25056]: cluster 2023-12-07T17:19:19.080247+0000 mgr.y (mgr.24626) 159 : cluster 0 pgmap v105: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:20.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:20 smithi064 bash[36207]: cluster 2023-12-07T17:19:19.080247+0000 mgr.y (mgr.24626) 159 : cluster 0 pgmap v105: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:19:22.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:21 smithi184 bash[31336]: cluster 2023-12-07T17:19:21.080857+0000 mgr.y (mgr.24626) 160 : cluster 0 pgmap v106: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:22.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:21 smithi184 bash[31336]: audit 2023-12-07T17:19:21.373434+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:19:22.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:21 smithi064 bash[25056]: cluster 2023-12-07T17:19:21.080857+0000 mgr.y (mgr.24626) 160 : cluster 0 pgmap v106: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:22.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:21 smithi064 bash[25056]: audit 2023-12-07T17:19:21.373434+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:22.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:21 smithi064 bash[36207]: cluster 2023-12-07T17:19:21.080857+0000 mgr.y (mgr.24626) 160 : cluster 0 pgmap v106: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:22.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:21 smithi064 bash[36207]: audit 2023-12-07T17:19:21.373434+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:24.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:24 smithi184 bash[31336]: cluster 2023-12-07T17:19:23.081460+0000 mgr.y (mgr.24626) 161 : cluster 0 pgmap v107: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:24.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:24 smithi064 bash[25056]: cluster 2023-12-07T17:19:23.081460+0000 mgr.y (mgr.24626) 161 : cluster 0 pgmap v107: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:24.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:24 smithi064 bash[36207]: cluster 2023-12-07T17:19:23.081460+0000 mgr.y (mgr.24626) 161 : cluster 0 pgmap v107: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:25.539 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:19:25.539 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:19:25.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:26.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:26 smithi184 bash[31336]: cluster 2023-12-07T17:19:25.081991+0000 mgr.y (mgr.24626) 162 : cluster 0 pgmap v108: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:26.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:26 smithi184 bash[31336]: audit 2023-12-07T17:19:25.536704+0000 mon.a (mon.0) 820 : audit 0 from='client.? 172.21.15.64:0/2557794816' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:26.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:26 smithi064 bash[36207]: cluster 2023-12-07T17:19:25.081991+0000 mgr.y (mgr.24626) 162 : cluster 0 pgmap v108: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:26.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:26 smithi064 bash[36207]: audit 2023-12-07T17:19:25.536704+0000 mon.a (mon.0) 820 : audit 0 from='client.? 172.21.15.64:0/2557794816' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:26.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:26 smithi064 bash[25056]: cluster 2023-12-07T17:19:25.081991+0000 mgr.y (mgr.24626) 162 : cluster 0 pgmap v108: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:26.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:26 smithi064 bash[25056]: audit 2023-12-07T17:19:25.536704+0000 mon.a (mon.0) 820 : audit 0 from='client.? 172.21.15.64:0/2557794816' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:27.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:28.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:28 smithi184 bash[31336]: cluster 2023-12-07T17:19:27.082784+0000 mgr.y (mgr.24626) 163 : cluster 0 pgmap v109: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:28.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:28 smithi184 bash[31336]: audit 2023-12-07T17:19:27.158876+0000 mgr.y (mgr.24626) 164 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:28.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:28 smithi064 bash[25056]: cluster 2023-12-07T17:19:27.082784+0000 mgr.y (mgr.24626) 163 : cluster 0 pgmap v109: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:28.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:28 smithi064 bash[25056]: audit 2023-12-07T17:19:27.158876+0000 mgr.y (mgr.24626) 164 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:28.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:28 smithi064 bash[36207]: cluster 2023-12-07T17:19:27.082784+0000 mgr.y (mgr.24626) 163 : cluster 0 pgmap v109: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:28.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:28 smithi064 bash[36207]: audit 2023-12-07T17:19:27.158876+0000 mgr.y (mgr.24626) 164 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:29.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:19:30.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:30 smithi184 bash[31336]: cluster 2023-12-07T17:19:29.083415+0000 mgr.y (mgr.24626) 165 : cluster 0 pgmap v110: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:30.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:30 smithi064 bash[25056]: cluster 2023-12-07T17:19:29.083415+0000 mgr.y (mgr.24626) 165 : cluster 0 pgmap v110: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:30.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:30 smithi064 bash[36207]: cluster 2023-12-07T17:19:29.083415+0000 mgr.y (mgr.24626) 165 : cluster 0 pgmap v110: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:32.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:31 smithi184 bash[31336]: cluster 2023-12-07T17:19:31.084067+0000 mgr.y (mgr.24626) 166 : cluster 0 pgmap v111: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:32.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:31 smithi064 bash[25056]: cluster 2023-12-07T17:19:31.084067+0000 mgr.y (mgr.24626) 166 : cluster 0 pgmap v111: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:32.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:31 smithi064 bash[36207]: cluster 2023-12-07T17:19:31.084067+0000 mgr.y (mgr.24626) 166 : cluster 0 pgmap v111: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:34.118 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:19:34.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:34 smithi184 bash[31336]: cluster 2023-12-07T17:19:33.084979+0000 mgr.y (mgr.24626) 167 : cluster 0 pgmap v112: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:34.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:34 smithi064 bash[25056]: cluster 2023-12-07T17:19:33.084979+0000 mgr.y (mgr.24626) 167 : cluster 0 pgmap v112: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:34.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:34 smithi064 bash[36207]: cluster 2023-12-07T17:19:33.084979+0000 mgr.y (mgr.24626) 167 : cluster 0 pgmap v112: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:35.953 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:35 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:36.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:36 smithi064 bash[25056]: cluster 2023-12-07T17:19:35.085653+0000 mgr.y (mgr.24626) 168 : cluster 0 pgmap v113: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:36.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:36 smithi064 bash[36207]: cluster 2023-12-07T17:19:35.085653+0000 mgr.y (mgr.24626) 168 : cluster 0 pgmap v113: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:36.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:36 smithi184 bash[31336]: cluster 2023-12-07T17:19:35.085653+0000 mgr.y (mgr.24626) 168 : cluster 0 pgmap v113: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:37.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:37 smithi064 bash[25056]: audit 2023-12-07T17:19:36.392135+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:37.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:37 smithi064 bash[36207]: audit 2023-12-07T17:19:36.392135+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:37.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:37 smithi184 bash[31336]: audit 2023-12-07T17:19:36.392135+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:37.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:37.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:19:37.593 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:19:38.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[25056]: cluster 2023-12-07T17:19:37.086545+0000 mgr.y (mgr.24626) 169 : cluster 0 pgmap v114: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:38.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[25056]: audit 2023-12-07T17:19:37.170049+0000 mgr.y (mgr.24626) 170 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:38.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[25056]: audit 2023-12-07T17:19:37.590585+0000 mon.c (mon.2) 84 : audit 0 from='client.? 172.21.15.64:0/243041415' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:38.452 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[36207]: cluster 2023-12-07T17:19:37.086545+0000 mgr.y (mgr.24626) 169 : cluster 0 pgmap v114: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:38.452 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[36207]: audit 2023-12-07T17:19:37.170049+0000 mgr.y (mgr.24626) 170 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:38.452 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:38 smithi064 bash[36207]: audit 2023-12-07T17:19:37.590585+0000 mon.c (mon.2) 84 : audit 0 from='client.? 172.21.15.64:0/243041415' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:38 smithi184 bash[31336]: cluster 2023-12-07T17:19:37.086545+0000 mgr.y (mgr.24626) 169 : cluster 0 pgmap v114: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:38 smithi184 bash[31336]: audit 2023-12-07T17:19:37.170049+0000 mgr.y (mgr.24626) 170 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:38 smithi184 bash[31336]: audit 2023-12-07T17:19:37.590585+0000 mon.c (mon.2) 84 : audit 0 from='client.? 172.21.15.64:0/243041415' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:40.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:40 smithi184 bash[31336]: cluster 2023-12-07T17:19:39.087140+0000 mgr.y (mgr.24626) 171 : cluster 0 pgmap v115: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:40.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:40 smithi064 bash[25056]: cluster 2023-12-07T17:19:39.087140+0000 mgr.y (mgr.24626) 171 : cluster 0 pgmap v115: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:40.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:40 smithi064 bash[36207]: cluster 2023-12-07T17:19:39.087140+0000 mgr.y (mgr.24626) 171 : cluster 0 pgmap v115: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:41.453 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:19:42.070 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:41 smithi184 bash[31336]: cluster 2023-12-07T17:19:41.087942+0000 mgr.y (mgr.24626) 172 : cluster 0 pgmap v116: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:42.070 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:41 smithi184 bash[31336]: audit 2023-12-07T17:19:41.171623+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:19:42.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:41 smithi064 bash[25056]: cluster 2023-12-07T17:19:41.087942+0000 mgr.y (mgr.24626) 172 : cluster 0 pgmap v116: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:42.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:41 smithi064 bash[25056]: audit 2023-12-07T17:19:41.171623+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:19:42.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:41 smithi064 bash[36207]: cluster 2023-12-07T17:19:41.087942+0000 mgr.y (mgr.24626) 172 : cluster 0 pgmap v116: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:42.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:41 smithi064 bash[36207]: audit 2023-12-07T17:19:41.171623+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:19:43.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:42 smithi184 bash[31336]: audit 2023-12-07T17:19:41.855697+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:19:43.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:42 smithi184 bash[31336]: audit 2023-12-07T17:19:41.857616+0000 mon.c (mon.2) 87 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:19:43.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:42 smithi184 bash[31336]: cluster 2023-12-07T17:19:41.860638+0000 mgr.y (mgr.24626) 173 : cluster 0 pgmap v117: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2023-12-07T17:19:43.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:42 smithi184 bash[31336]: cluster 2023-12-07T17:19:41.861033+0000 mgr.y (mgr.24626) 174 : cluster 0 pgmap v118: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:43.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:42 smithi184 bash[31336]: audit 2023-12-07T17:19:41.868146+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:19:43.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[25056]: audit 2023-12-07T17:19:41.855697+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:19:43.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[25056]: audit 2023-12-07T17:19:41.857616+0000 mon.c (mon.2) 87 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:19:43.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[25056]: cluster 2023-12-07T17:19:41.860638+0000 mgr.y (mgr.24626) 173 : cluster 0 pgmap v117: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2023-12-07T17:19:43.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[25056]: cluster 2023-12-07T17:19:41.861033+0000 mgr.y (mgr.24626) 174 : cluster 0 pgmap v118: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[25056]: audit 2023-12-07T17:19:41.868146+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[36207]: audit 2023-12-07T17:19:41.855697+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[36207]: audit 2023-12-07T17:19:41.857616+0000 mon.c (mon.2) 87 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[36207]: cluster 2023-12-07T17:19:41.860638+0000 mgr.y (mgr.24626) 173 : cluster 0 pgmap v117: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[36207]: cluster 2023-12-07T17:19:41.861033+0000 mgr.y (mgr.24626) 174 : cluster 0 pgmap v118: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:43.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:42 smithi064 bash[36207]: audit 2023-12-07T17:19:41.868146+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:19:45.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:44 smithi184 bash[31336]: cluster 2023-12-07T17:19:43.861834+0000 mgr.y (mgr.24626) 175 : cluster 0 pgmap v119: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:45.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:44 smithi064 bash[25056]: cluster 2023-12-07T17:19:43.861834+0000 mgr.y (mgr.24626) 175 : cluster 0 pgmap v119: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:44 smithi064 bash[36207]: cluster 2023-12-07T17:19:43.861834+0000 mgr.y (mgr.24626) 175 : cluster 0 pgmap v119: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:46.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:19:47.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:46 smithi184 bash[31336]: cluster 2023-12-07T17:19:45.862499+0000 mgr.y (mgr.24626) 176 : cluster 0 pgmap v120: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:47.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:46 smithi064 bash[36207]: cluster 2023-12-07T17:19:45.862499+0000 mgr.y (mgr.24626) 176 : cluster 0 pgmap v120: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:47.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:46 smithi064 bash[25056]: cluster 2023-12-07T17:19:45.862499+0000 mgr.y (mgr.24626) 176 : cluster 0 pgmap v120: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2023-12-07T17:19:47.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:48.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:47 smithi184 bash[31336]: audit 2023-12-07T17:19:47.181287+0000 mgr.y (mgr.24626) 177 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:48.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:47 smithi064 bash[25056]: audit 2023-12-07T17:19:47.181287+0000 mgr.y (mgr.24626) 177 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:48.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:47 smithi064 bash[36207]: audit 2023-12-07T17:19:47.181287+0000 mgr.y (mgr.24626) 177 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:49.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:48 smithi184 bash[31336]: cluster 2023-12-07T17:19:47.863364+0000 mgr.y (mgr.24626) 178 : cluster 0 pgmap v121: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:49.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:48 smithi064 bash[25056]: cluster 2023-12-07T17:19:47.863364+0000 mgr.y (mgr.24626) 178 : cluster 0 pgmap v121: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:49.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:48 smithi064 bash[36207]: cluster 2023-12-07T17:19:47.863364+0000 mgr.y (mgr.24626) 178 : cluster 0 pgmap v121: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:49.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:19:49.647 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:19:50.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:49 smithi184 bash[31336]: audit 2023-12-07T17:19:49.644340+0000 mon.a (mon.0) 822 : audit 0 from='client.? 172.21.15.64:0/3838669768' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:50.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:49 smithi064 bash[25056]: audit 2023-12-07T17:19:49.644340+0000 mon.a (mon.0) 822 : audit 0 from='client.? 172.21.15.64:0/3838669768' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:50.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:49 smithi064 bash[36207]: audit 2023-12-07T17:19:49.644340+0000 mon.a (mon.0) 822 : audit 0 from='client.? 172.21.15.64:0/3838669768' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:19:51.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:50 smithi184 bash[31336]: cluster 2023-12-07T17:19:49.863927+0000 mgr.y (mgr.24626) 179 : cluster 0 pgmap v122: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:51.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:50 smithi064 bash[25056]: cluster 2023-12-07T17:19:49.863927+0000 mgr.y (mgr.24626) 179 : cluster 0 pgmap v122: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:51.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:50 smithi064 bash[36207]: cluster 2023-12-07T17:19:49.863927+0000 mgr.y (mgr.24626) 179 : cluster 0 pgmap v122: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:19:52.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:51 smithi184 bash[31336]: audit 2023-12-07T17:19:51.408104+0000 mon.c (mon.2) 88 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:52.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:51 smithi064 bash[25056]: audit 2023-12-07T17:19:51.408104+0000 mon.c (mon.2) 88 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:52.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:51 smithi064 bash[36207]: audit 2023-12-07T17:19:51.408104+0000 mon.c (mon.2) 88 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:19:53.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:52 smithi064 bash[25056]: cluster 2023-12-07T17:19:51.864538+0000 mgr.y (mgr.24626) 180 : cluster 0 pgmap v123: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:19:53.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:52 smithi064 bash[36207]: cluster 2023-12-07T17:19:51.864538+0000 mgr.y (mgr.24626) 180 : cluster 0 pgmap v123: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:19:53.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:52 smithi184 bash[31336]: cluster 2023-12-07T17:19:51.864538+0000 mgr.y (mgr.24626) 180 : cluster 0 pgmap v123: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:19:53.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:19:55.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:54 smithi184 bash[31336]: cluster 2023-12-07T17:19:53.865508+0000 mgr.y (mgr.24626) 181 : cluster 0 pgmap v124: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:55.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:54 smithi064 bash[25056]: cluster 2023-12-07T17:19:53.865508+0000 mgr.y (mgr.24626) 181 : cluster 0 pgmap v124: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:55.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:54 smithi064 bash[36207]: cluster 2023-12-07T17:19:53.865508+0000 mgr.y (mgr.24626) 181 : cluster 0 pgmap v124: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:56.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:19:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:19:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:19:57.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:56 smithi184 bash[31336]: cluster 2023-12-07T17:19:55.866118+0000 mgr.y (mgr.24626) 182 : cluster 0 pgmap v125: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:57.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:56 smithi064 bash[25056]: cluster 2023-12-07T17:19:55.866118+0000 mgr.y (mgr.24626) 182 : cluster 0 pgmap v125: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:57.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:56 smithi064 bash[36207]: cluster 2023-12-07T17:19:55.866118+0000 mgr.y (mgr.24626) 182 : cluster 0 pgmap v125: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:19:57.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:19:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:19:58.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:57 smithi184 bash[31336]: audit 2023-12-07T17:19:57.190814+0000 mgr.y (mgr.24626) 183 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:58.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:57 smithi064 bash[36207]: audit 2023-12-07T17:19:57.190814+0000 mgr.y (mgr.24626) 183 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:58.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:57 smithi064 bash[25056]: audit 2023-12-07T17:19:57.190814+0000 mgr.y (mgr.24626) 183 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:19:58.233 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:19:59.067 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:19:58 smithi184 bash[31336]: cluster 2023-12-07T17:19:57.866993+0000 mgr.y (mgr.24626) 184 : cluster 0 pgmap v126: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:59.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:19:58 smithi064 bash[25056]: cluster 2023-12-07T17:19:57.866993+0000 mgr.y (mgr.24626) 184 : cluster 0 pgmap v126: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:19:59.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:19:58 smithi064 bash[36207]: cluster 2023-12-07T17:19:57.866993+0000 mgr.y (mgr.24626) 184 : cluster 0 pgmap v126: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:01.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[25056]: cluster 2023-12-07T17:19:59.867611+0000 mgr.y (mgr.24626) 185 : cluster 0 pgmap v127: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:01.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[25056]: cluster 2023-12-07T17:20:00.000124+0000 mon.a (mon.0) 823 : cluster 3 Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-12-07T17:20:01.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[25056]: cluster 2023-12-07T17:20:00.000172+0000 mon.a (mon.0) 824 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-12-07T17:20:01.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[25056]: cluster 2023-12-07T17:20:00.000192+0000 mon.a (mon.0) 825 : cluster 3 daemon rgw.foo.a on smithi064 is in error state 2023-12-07T17:20:01.200 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[36207]: cluster 2023-12-07T17:19:59.867611+0000 mgr.y (mgr.24626) 185 : cluster 0 pgmap v127: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:01.200 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[36207]: cluster 2023-12-07T17:20:00.000124+0000 mon.a (mon.0) 823 : cluster 3 Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-12-07T17:20:01.200 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[36207]: cluster 2023-12-07T17:20:00.000172+0000 mon.a (mon.0) 824 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-12-07T17:20:01.200 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:00 smithi064 bash[36207]: cluster 2023-12-07T17:20:00.000192+0000 mon.a (mon.0) 825 : cluster 3 daemon rgw.foo.a on smithi064 is in error state 2023-12-07T17:20:01.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:00 smithi184 bash[31336]: cluster 2023-12-07T17:19:59.867611+0000 mgr.y (mgr.24626) 185 : cluster 0 pgmap v127: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:01.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:00 smithi184 bash[31336]: cluster 2023-12-07T17:20:00.000124+0000 mon.a (mon.0) 823 : cluster 3 Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2023-12-07T17:20:01.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:00 smithi184 bash[31336]: cluster 2023-12-07T17:20:00.000172+0000 mon.a (mon.0) 824 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2023-12-07T17:20:01.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:00 smithi184 bash[31336]: cluster 2023-12-07T17:20:00.000192+0000 mon.a (mon.0) 825 : cluster 3 daemon rgw.foo.a on smithi064 is in error state 2023-12-07T17:20:01.638 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:20:01.638 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:20:02.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:01 smithi064 bash[36207]: audit 2023-12-07T17:20:01.635625+0000 mon.c (mon.2) 89 : audit 0 from='client.? 172.21.15.64:0/2647089423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:02.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:01 smithi064 bash[25056]: audit 2023-12-07T17:20:01.635625+0000 mon.c (mon.2) 89 : audit 0 from='client.? 172.21.15.64:0/2647089423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:02.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:01 smithi184 bash[31336]: audit 2023-12-07T17:20:01.635625+0000 mon.c (mon.2) 89 : audit 0 from='client.? 172.21.15.64:0/2647089423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:03.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:02 smithi184 bash[31336]: cluster 2023-12-07T17:20:01.868249+0000 mgr.y (mgr.24626) 186 : cluster 0 pgmap v128: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:03.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:02 smithi064 bash[25056]: cluster 2023-12-07T17:20:01.868249+0000 mgr.y (mgr.24626) 186 : cluster 0 pgmap v128: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:03.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:02 smithi064 bash[36207]: cluster 2023-12-07T17:20:01.868249+0000 mgr.y (mgr.24626) 186 : cluster 0 pgmap v128: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:05.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:04 smithi184 bash[31336]: cluster 2023-12-07T17:20:03.869112+0000 mgr.y (mgr.24626) 187 : cluster 0 pgmap v129: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:05.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:04 smithi064 bash[25056]: cluster 2023-12-07T17:20:03.869112+0000 mgr.y (mgr.24626) 187 : cluster 0 pgmap v129: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:05.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:04 smithi064 bash[36207]: cluster 2023-12-07T17:20:03.869112+0000 mgr.y (mgr.24626) 187 : cluster 0 pgmap v129: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:05.583 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:20:05.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:07.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:06 smithi184 bash[31336]: cluster 2023-12-07T17:20:05.869702+0000 mgr.y (mgr.24626) 188 : cluster 0 pgmap v130: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:07.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:06 smithi184 bash[31336]: audit 2023-12-07T17:20:06.426706+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:06 smithi064 bash[25056]: cluster 2023-12-07T17:20:05.869702+0000 mgr.y (mgr.24626) 188 : cluster 0 pgmap v130: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:07.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:06 smithi064 bash[25056]: audit 2023-12-07T17:20:06.426706+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:07.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:06 smithi064 bash[36207]: cluster 2023-12-07T17:20:05.869702+0000 mgr.y (mgr.24626) 188 : cluster 0 pgmap v130: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:07.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:06 smithi064 bash[36207]: audit 2023-12-07T17:20:06.426706+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:07.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:08.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:07 smithi184 bash[31336]: audit 2023-12-07T17:20:07.202109+0000 mgr.y (mgr.24626) 189 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:08.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:07 smithi064 bash[25056]: audit 2023-12-07T17:20:07.202109+0000 mgr.y (mgr.24626) 189 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:08.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:07 smithi064 bash[36207]: audit 2023-12-07T17:20:07.202109+0000 mgr.y (mgr.24626) 189 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:09.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:08 smithi184 bash[31336]: cluster 2023-12-07T17:20:07.870571+0000 mgr.y (mgr.24626) 190 : cluster 0 pgmap v131: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:09.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:08 smithi064 bash[25056]: cluster 2023-12-07T17:20:07.870571+0000 mgr.y (mgr.24626) 190 : cluster 0 pgmap v131: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:09.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:08 smithi064 bash[36207]: cluster 2023-12-07T17:20:07.870571+0000 mgr.y (mgr.24626) 190 : cluster 0 pgmap v131: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:10.362 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:20:11.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:10 smithi184 bash[31336]: cluster 2023-12-07T17:20:09.871237+0000 mgr.y (mgr.24626) 191 : cluster 0 pgmap v132: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:11.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:10 smithi064 bash[25056]: cluster 2023-12-07T17:20:09.871237+0000 mgr.y (mgr.24626) 191 : cluster 0 pgmap v132: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:11.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:10 smithi064 bash[36207]: cluster 2023-12-07T17:20:09.871237+0000 mgr.y (mgr.24626) 191 : cluster 0 pgmap v132: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:13.237 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:12 smithi064 bash[25056]: cluster 2023-12-07T17:20:11.871850+0000 mgr.y (mgr.24626) 192 : cluster 0 pgmap v133: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:13.238 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:12 smithi064 bash[36207]: cluster 2023-12-07T17:20:11.871850+0000 mgr.y (mgr.24626) 192 : cluster 0 pgmap v133: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:13.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:12 smithi184 bash[31336]: cluster 2023-12-07T17:20:11.871850+0000 mgr.y (mgr.24626) 192 : cluster 0 pgmap v133: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:13.782 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:20:13.782 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:20:14.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:13 smithi064 bash[36207]: audit 2023-12-07T17:20:13.779463+0000 mon.c (mon.2) 91 : audit 0 from='client.? 172.21.15.64:0/3488219419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:14.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:13 smithi064 bash[25056]: audit 2023-12-07T17:20:13.779463+0000 mon.c (mon.2) 91 : audit 0 from='client.? 172.21.15.64:0/3488219419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:14.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:13 smithi184 bash[31336]: audit 2023-12-07T17:20:13.779463+0000 mon.c (mon.2) 91 : audit 0 from='client.? 172.21.15.64:0/3488219419' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:15.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:14 smithi184 bash[31336]: cluster 2023-12-07T17:20:13.872776+0000 mgr.y (mgr.24626) 193 : cluster 0 pgmap v134: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:15.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:14 smithi064 bash[25056]: cluster 2023-12-07T17:20:13.872776+0000 mgr.y (mgr.24626) 193 : cluster 0 pgmap v134: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:15.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:14 smithi064 bash[36207]: cluster 2023-12-07T17:20:13.872776+0000 mgr.y (mgr.24626) 193 : cluster 0 pgmap v134: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:16.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:17.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:16 smithi184 bash[31336]: cluster 2023-12-07T17:20:15.873422+0000 mgr.y (mgr.24626) 194 : cluster 0 pgmap v135: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:17.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:16 smithi064 bash[25056]: cluster 2023-12-07T17:20:15.873422+0000 mgr.y (mgr.24626) 194 : cluster 0 pgmap v135: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:17.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:16 smithi064 bash[36207]: cluster 2023-12-07T17:20:15.873422+0000 mgr.y (mgr.24626) 194 : cluster 0 pgmap v135: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:17.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:20:17.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:17.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:17 smithi064 bash[25056]: audit 2023-12-07T17:20:17.213458+0000 mgr.y (mgr.24626) 195 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:17.870 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:17 smithi064 bash[36207]: audit 2023-12-07T17:20:17.213458+0000 mgr.y (mgr.24626) 195 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:18.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:17 smithi184 bash[31336]: audit 2023-12-07T17:20:17.213458+0000 mgr.y (mgr.24626) 195 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:19.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:18 smithi064 bash[25056]: cluster 2023-12-07T17:20:17.874323+0000 mgr.y (mgr.24626) 196 : cluster 0 pgmap v136: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:19.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:18 smithi064 bash[36207]: cluster 2023-12-07T17:20:17.874323+0000 mgr.y (mgr.24626) 196 : cluster 0 pgmap v136: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:19.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:18 smithi184 bash[31336]: cluster 2023-12-07T17:20:17.874323+0000 mgr.y (mgr.24626) 196 : cluster 0 pgmap v136: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:21.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:20 smithi184 bash[31336]: cluster 2023-12-07T17:20:19.875027+0000 mgr.y (mgr.24626) 197 : cluster 0 pgmap v137: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:21.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:20 smithi064 bash[25056]: cluster 2023-12-07T17:20:19.875027+0000 mgr.y (mgr.24626) 197 : cluster 0 pgmap v137: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:21.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:20 smithi064 bash[36207]: cluster 2023-12-07T17:20:19.875027+0000 mgr.y (mgr.24626) 197 : cluster 0 pgmap v137: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:22.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:21 smithi184 bash[31336]: audit 2023-12-07T17:20:21.446458+0000 mon.c (mon.2) 92 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:22.334 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:20:22.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:21 smithi064 bash[25056]: audit 2023-12-07T17:20:21.446458+0000 mon.c (mon.2) 92 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:22.367 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:21 smithi064 bash[36207]: audit 2023-12-07T17:20:21.446458+0000 mon.c (mon.2) 92 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:23.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:22 smithi184 bash[31336]: cluster 2023-12-07T17:20:21.875874+0000 mgr.y (mgr.24626) 198 : cluster 0 pgmap v138: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:23.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:22 smithi064 bash[25056]: cluster 2023-12-07T17:20:21.875874+0000 mgr.y (mgr.24626) 198 : cluster 0 pgmap v138: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:23.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:22 smithi064 bash[36207]: cluster 2023-12-07T17:20:21.875874+0000 mgr.y (mgr.24626) 198 : cluster 0 pgmap v138: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:25.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:24 smithi184 bash[31336]: cluster 2023-12-07T17:20:23.876754+0000 mgr.y (mgr.24626) 199 : cluster 0 pgmap v139: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:25.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:24 smithi064 bash[25056]: cluster 2023-12-07T17:20:23.876754+0000 mgr.y (mgr.24626) 199 : cluster 0 pgmap v139: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:25.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:24 smithi064 bash[36207]: cluster 2023-12-07T17:20:23.876754+0000 mgr.y (mgr.24626) 199 : cluster 0 pgmap v139: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:25.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:20:25.872 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:20:26.075 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:27.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:26 smithi184 bash[31336]: audit 2023-12-07T17:20:25.868845+0000 mon.c (mon.2) 93 : audit 0 from='client.? 172.21.15.64:0/3836918912' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:27.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:26 smithi184 bash[31336]: cluster 2023-12-07T17:20:25.877334+0000 mgr.y (mgr.24626) 200 : cluster 0 pgmap v140: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:27.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:26 smithi064 bash[25056]: audit 2023-12-07T17:20:25.868845+0000 mon.c (mon.2) 93 : audit 0 from='client.? 172.21.15.64:0/3836918912' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:27.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:26 smithi064 bash[25056]: cluster 2023-12-07T17:20:25.877334+0000 mgr.y (mgr.24626) 200 : cluster 0 pgmap v140: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:27.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:26 smithi064 bash[36207]: audit 2023-12-07T17:20:25.868845+0000 mon.c (mon.2) 93 : audit 0 from='client.? 172.21.15.64:0/3836918912' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:27.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:26 smithi064 bash[36207]: cluster 2023-12-07T17:20:25.877334+0000 mgr.y (mgr.24626) 200 : cluster 0 pgmap v140: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:27.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:28.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:27 smithi184 bash[31336]: audit 2023-12-07T17:20:27.223244+0000 mgr.y (mgr.24626) 201 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:28.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:27 smithi064 bash[25056]: audit 2023-12-07T17:20:27.223244+0000 mgr.y (mgr.24626) 201 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:28.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:27 smithi064 bash[36207]: audit 2023-12-07T17:20:27.223244+0000 mgr.y (mgr.24626) 201 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:29.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:28 smithi184 bash[31336]: cluster 2023-12-07T17:20:27.878212+0000 mgr.y (mgr.24626) 202 : cluster 0 pgmap v141: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:28 smithi064 bash[25056]: cluster 2023-12-07T17:20:27.878212+0000 mgr.y (mgr.24626) 202 : cluster 0 pgmap v141: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:29.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:28 smithi064 bash[36207]: cluster 2023-12-07T17:20:27.878212+0000 mgr.y (mgr.24626) 202 : cluster 0 pgmap v141: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:29.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:20:31.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:30 smithi184 bash[31336]: cluster 2023-12-07T17:20:29.878852+0000 mgr.y (mgr.24626) 203 : cluster 0 pgmap v142: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:31.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:30 smithi064 bash[25056]: cluster 2023-12-07T17:20:29.878852+0000 mgr.y (mgr.24626) 203 : cluster 0 pgmap v142: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:31.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:30 smithi064 bash[36207]: cluster 2023-12-07T17:20:29.878852+0000 mgr.y (mgr.24626) 203 : cluster 0 pgmap v142: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:20:33.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:32 smithi184 bash[31336]: cluster 2023-12-07T17:20:31.879481+0000 mgr.y (mgr.24626) 204 : cluster 0 pgmap v143: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:33.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:32 smithi064 bash[25056]: cluster 2023-12-07T17:20:31.879481+0000 mgr.y (mgr.24626) 204 : cluster 0 pgmap v143: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:33.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:32 smithi064 bash[36207]: cluster 2023-12-07T17:20:31.879481+0000 mgr.y (mgr.24626) 204 : cluster 0 pgmap v143: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:20:35.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:34 smithi184 bash[31336]: cluster 2023-12-07T17:20:33.880362+0000 mgr.y (mgr.24626) 205 : cluster 0 pgmap v144: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:35.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:34 smithi064 bash[25056]: cluster 2023-12-07T17:20:33.880362+0000 mgr.y (mgr.24626) 205 : cluster 0 pgmap v144: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:35.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:34 smithi064 bash[36207]: cluster 2023-12-07T17:20:33.880362+0000 mgr.y (mgr.24626) 205 : cluster 0 pgmap v144: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:36.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:35 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:37.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:36 smithi184 bash[31336]: cluster 2023-12-07T17:20:35.880904+0000 mgr.y (mgr.24626) 206 : cluster 0 pgmap v145: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:37.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:36 smithi184 bash[31336]: audit 2023-12-07T17:20:36.464290+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:37.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:36 smithi064 bash[36207]: cluster 2023-12-07T17:20:35.880904+0000 mgr.y (mgr.24626) 206 : cluster 0 pgmap v145: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:37.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:36 smithi064 bash[36207]: audit 2023-12-07T17:20:36.464290+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:36 smithi064 bash[25056]: cluster 2023-12-07T17:20:35.880904+0000 mgr.y (mgr.24626) 206 : cluster 0 pgmap v145: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:36 smithi064 bash[25056]: audit 2023-12-07T17:20:36.464290+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:37.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:37.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:20:37.777 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:20:38.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:37 smithi184 bash[31336]: audit 2023-12-07T17:20:37.231705+0000 mgr.y (mgr.24626) 207 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:38.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:37 smithi184 bash[31336]: audit 2023-12-07T17:20:37.774192+0000 mon.c (mon.2) 95 : audit 0 from='client.? 172.21.15.64:0/4000637281' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:38.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:37 smithi064 bash[36207]: audit 2023-12-07T17:20:37.231705+0000 mgr.y (mgr.24626) 207 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:38.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:37 smithi064 bash[36207]: audit 2023-12-07T17:20:37.774192+0000 mon.c (mon.2) 95 : audit 0 from='client.? 172.21.15.64:0/4000637281' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:38.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:37 smithi064 bash[25056]: audit 2023-12-07T17:20:37.231705+0000 mgr.y (mgr.24626) 207 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:38.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:37 smithi064 bash[25056]: audit 2023-12-07T17:20:37.774192+0000 mon.c (mon.2) 95 : audit 0 from='client.? 172.21.15.64:0/4000637281' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:38.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:38 smithi064 bash[25056]: cluster 2023-12-07T17:20:37.881725+0000 mgr.y (mgr.24626) 208 : cluster 0 pgmap v146: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:38.871 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:38 smithi064 bash[36207]: cluster 2023-12-07T17:20:37.881725+0000 mgr.y (mgr.24626) 208 : cluster 0 pgmap v146: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:39.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:38 smithi184 bash[31336]: cluster 2023-12-07T17:20:37.881725+0000 mgr.y (mgr.24626) 208 : cluster 0 pgmap v146: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:20:41.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:40 smithi184 bash[31336]: cluster 2023-12-07T17:20:39.882351+0000 mgr.y (mgr.24626) 209 : cluster 0 pgmap v147: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:41.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:40 smithi064 bash[25056]: cluster 2023-12-07T17:20:39.882351+0000 mgr.y (mgr.24626) 209 : cluster 0 pgmap v147: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:41.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:40 smithi064 bash[36207]: cluster 2023-12-07T17:20:39.882351+0000 mgr.y (mgr.24626) 209 : cluster 0 pgmap v147: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:41.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:20:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:42 smithi184 bash[31336]: cluster 2023-12-07T17:20:41.882993+0000 mgr.y (mgr.24626) 210 : cluster 0 pgmap v148: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:42 smithi184 bash[31336]: audit 2023-12-07T17:20:41.950593+0000 mon.c (mon.2) 96 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:20:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:42 smithi184 bash[31336]: audit 2023-12-07T17:20:42.635192+0000 mon.c (mon.2) 97 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:20:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:42 smithi184 bash[31336]: audit 2023-12-07T17:20:42.637023+0000 mon.c (mon.2) 98 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:20:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:42 smithi184 bash[31336]: audit 2023-12-07T17:20:42.646789+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:20:43.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[25056]: cluster 2023-12-07T17:20:41.882993+0000 mgr.y (mgr.24626) 210 : cluster 0 pgmap v148: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[25056]: audit 2023-12-07T17:20:41.950593+0000 mon.c (mon.2) 96 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:20:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[25056]: audit 2023-12-07T17:20:42.635192+0000 mon.c (mon.2) 97 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:20:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[25056]: audit 2023-12-07T17:20:42.637023+0000 mon.c (mon.2) 98 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:20:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[25056]: audit 2023-12-07T17:20:42.646789+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:20:43.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[36207]: cluster 2023-12-07T17:20:41.882993+0000 mgr.y (mgr.24626) 210 : cluster 0 pgmap v148: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:43.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[36207]: audit 2023-12-07T17:20:41.950593+0000 mon.c (mon.2) 96 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:20:43.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[36207]: audit 2023-12-07T17:20:42.635192+0000 mon.c (mon.2) 97 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:20:43.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[36207]: audit 2023-12-07T17:20:42.637023+0000 mon.c (mon.2) 98 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:20:43.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:42 smithi064 bash[36207]: audit 2023-12-07T17:20:42.646789+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:20:44.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:43 smithi184 bash[31336]: cluster 2023-12-07T17:20:42.640202+0000 mgr.y (mgr.24626) 211 : cluster 0 pgmap v149: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:44.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:43 smithi064 bash[36207]: cluster 2023-12-07T17:20:42.640202+0000 mgr.y (mgr.24626) 211 : cluster 0 pgmap v149: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:43 smithi064 bash[25056]: cluster 2023-12-07T17:20:42.640202+0000 mgr.y (mgr.24626) 211 : cluster 0 pgmap v149: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:45.959 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:46.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:45 smithi184 bash[31336]: cluster 2023-12-07T17:20:44.640958+0000 mgr.y (mgr.24626) 212 : cluster 0 pgmap v150: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:46.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:45 smithi064 bash[36207]: cluster 2023-12-07T17:20:44.640958+0000 mgr.y (mgr.24626) 212 : cluster 0 pgmap v150: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:46.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:45 smithi064 bash[25056]: cluster 2023-12-07T17:20:44.640958+0000 mgr.y (mgr.24626) 212 : cluster 0 pgmap v150: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:20:47.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:48.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:47 smithi064 bash[25056]: cluster 2023-12-07T17:20:46.641508+0000 mgr.y (mgr.24626) 213 : cluster 0 pgmap v151: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:48.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:47 smithi064 bash[25056]: audit 2023-12-07T17:20:47.243142+0000 mgr.y (mgr.24626) 214 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:48.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:47 smithi064 bash[36207]: cluster 2023-12-07T17:20:46.641508+0000 mgr.y (mgr.24626) 213 : cluster 0 pgmap v151: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:48.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:47 smithi064 bash[36207]: audit 2023-12-07T17:20:47.243142+0000 mgr.y (mgr.24626) 214 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:48.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:47 smithi184 bash[31336]: cluster 2023-12-07T17:20:46.641508+0000 mgr.y (mgr.24626) 213 : cluster 0 pgmap v151: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:48.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:47 smithi184 bash[31336]: audit 2023-12-07T17:20:47.243142+0000 mgr.y (mgr.24626) 214 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:49.908 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:20:49.908 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:20:50.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:49 smithi184 bash[31336]: cluster 2023-12-07T17:20:48.642232+0000 mgr.y (mgr.24626) 215 : cluster 0 pgmap v152: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:50.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:49 smithi184 bash[31336]: audit 2023-12-07T17:20:49.905144+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.64:0/3875800411' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:50.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:49 smithi064 bash[25056]: cluster 2023-12-07T17:20:48.642232+0000 mgr.y (mgr.24626) 215 : cluster 0 pgmap v152: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:50.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:49 smithi064 bash[25056]: audit 2023-12-07T17:20:49.905144+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.64:0/3875800411' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:50.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:49 smithi064 bash[36207]: cluster 2023-12-07T17:20:48.642232+0000 mgr.y (mgr.24626) 215 : cluster 0 pgmap v152: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:50.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:49 smithi064 bash[36207]: audit 2023-12-07T17:20:49.905144+0000 mon.a (mon.0) 827 : audit 0 from='client.? 172.21.15.64:0/3875800411' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:20:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:51 smithi184 bash[31336]: cluster 2023-12-07T17:20:50.642874+0000 mgr.y (mgr.24626) 216 : cluster 0 pgmap v153: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:52.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:51 smithi184 bash[31336]: audit 2023-12-07T17:20:51.482702+0000 mon.c (mon.2) 99 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:52.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:51 smithi064 bash[25056]: cluster 2023-12-07T17:20:50.642874+0000 mgr.y (mgr.24626) 216 : cluster 0 pgmap v153: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:52.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:51 smithi064 bash[25056]: audit 2023-12-07T17:20:51.482702+0000 mon.c (mon.2) 99 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:52.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:51 smithi064 bash[36207]: cluster 2023-12-07T17:20:50.642874+0000 mgr.y (mgr.24626) 216 : cluster 0 pgmap v153: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2023-12-07T17:20:52.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:51 smithi064 bash[36207]: audit 2023-12-07T17:20:51.482702+0000 mon.c (mon.2) 99 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:20:53.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:20:54.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:53 smithi184 bash[31336]: cluster 2023-12-07T17:20:52.643763+0000 mgr.y (mgr.24626) 217 : cluster 0 pgmap v154: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:54.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:53 smithi064 bash[25056]: cluster 2023-12-07T17:20:52.643763+0000 mgr.y (mgr.24626) 217 : cluster 0 pgmap v154: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:54.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:53 smithi064 bash[36207]: cluster 2023-12-07T17:20:52.643763+0000 mgr.y (mgr.24626) 217 : cluster 0 pgmap v154: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2023-12-07T17:20:55.979 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:20:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:20:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:20:56.317 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:55 smithi184 bash[31336]: cluster 2023-12-07T17:20:54.644387+0000 mgr.y (mgr.24626) 218 : cluster 0 pgmap v155: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:56.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:55 smithi064 bash[25056]: cluster 2023-12-07T17:20:54.644387+0000 mgr.y (mgr.24626) 218 : cluster 0 pgmap v155: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:56.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:55 smithi064 bash[36207]: cluster 2023-12-07T17:20:54.644387+0000 mgr.y (mgr.24626) 218 : cluster 0 pgmap v155: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:57.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:20:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:20:58.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:57 smithi184 bash[31336]: cluster 2023-12-07T17:20:56.645058+0000 mgr.y (mgr.24626) 219 : cluster 0 pgmap v156: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:58.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:57 smithi184 bash[31336]: audit 2023-12-07T17:20:57.247500+0000 mgr.y (mgr.24626) 220 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:58.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:57 smithi064 bash[25056]: cluster 2023-12-07T17:20:56.645058+0000 mgr.y (mgr.24626) 219 : cluster 0 pgmap v156: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:58.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:57 smithi064 bash[25056]: audit 2023-12-07T17:20:57.247500+0000 mgr.y (mgr.24626) 220 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:58.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:57 smithi064 bash[36207]: cluster 2023-12-07T17:20:56.645058+0000 mgr.y (mgr.24626) 219 : cluster 0 pgmap v156: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:20:58.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:57 smithi064 bash[36207]: audit 2023-12-07T17:20:57.247500+0000 mgr.y (mgr.24626) 220 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:20:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:00.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:20:59 smithi184 bash[31336]: cluster 2023-12-07T17:20:58.645603+0000 mgr.y (mgr.24626) 221 : cluster 0 pgmap v157: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:00.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:20:59 smithi064 bash[25056]: cluster 2023-12-07T17:20:58.645603+0000 mgr.y (mgr.24626) 221 : cluster 0 pgmap v157: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:00.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:20:59 smithi064 bash[36207]: cluster 2023-12-07T17:20:58.645603+0000 mgr.y (mgr.24626) 221 : cluster 0 pgmap v157: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:01.875 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:21:01.876 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:21:02.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:01 smithi184 bash[31336]: cluster 2023-12-07T17:21:00.646236+0000 mgr.y (mgr.24626) 222 : cluster 0 pgmap v158: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:02.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:01 smithi184 bash[31336]: audit 2023-12-07T17:21:01.872699+0000 mon.c (mon.2) 100 : audit 0 from='client.? 172.21.15.64:0/2724631023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:02.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:01 smithi064 bash[25056]: cluster 2023-12-07T17:21:00.646236+0000 mgr.y (mgr.24626) 222 : cluster 0 pgmap v158: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:02.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:01 smithi064 bash[25056]: audit 2023-12-07T17:21:01.872699+0000 mon.c (mon.2) 100 : audit 0 from='client.? 172.21.15.64:0/2724631023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:02.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:01 smithi064 bash[36207]: cluster 2023-12-07T17:21:00.646236+0000 mgr.y (mgr.24626) 222 : cluster 0 pgmap v158: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:02.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:01 smithi064 bash[36207]: audit 2023-12-07T17:21:01.872699+0000 mon.c (mon.2) 100 : audit 0 from='client.? 172.21.15.64:0/2724631023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:04.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:03 smithi184 bash[31336]: cluster 2023-12-07T17:21:02.647240+0000 mgr.y (mgr.24626) 223 : cluster 0 pgmap v159: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:04.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:03 smithi064 bash[25056]: cluster 2023-12-07T17:21:02.647240+0000 mgr.y (mgr.24626) 223 : cluster 0 pgmap v159: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:04.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:03 smithi064 bash[36207]: cluster 2023-12-07T17:21:02.647240+0000 mgr.y (mgr.24626) 223 : cluster 0 pgmap v159: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:05.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:21:06.003 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:06.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:06 smithi184 bash[31336]: cluster 2023-12-07T17:21:04.648059+0000 mgr.y (mgr.24626) 224 : cluster 0 pgmap v160: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:06.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:06 smithi064 bash[25056]: cluster 2023-12-07T17:21:04.648059+0000 mgr.y (mgr.24626) 224 : cluster 0 pgmap v160: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:06.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:06 smithi064 bash[36207]: cluster 2023-12-07T17:21:04.648059+0000 mgr.y (mgr.24626) 224 : cluster 0 pgmap v160: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:07.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:07 smithi184 bash[31336]: audit 2023-12-07T17:21:06.503657+0000 mon.c (mon.2) 101 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:07.318 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:07.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:07 smithi064 bash[25056]: audit 2023-12-07T17:21:06.503657+0000 mon.c (mon.2) 101 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:07.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:07 smithi064 bash[36207]: audit 2023-12-07T17:21:06.503657+0000 mon.c (mon.2) 101 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:08 smithi184 bash[31336]: cluster 2023-12-07T17:21:06.648586+0000 mgr.y (mgr.24626) 225 : cluster 0 pgmap v161: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:08.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:08 smithi184 bash[31336]: audit 2023-12-07T17:21:07.257350+0000 mgr.y (mgr.24626) 226 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:08 smithi064 bash[25056]: cluster 2023-12-07T17:21:06.648586+0000 mgr.y (mgr.24626) 225 : cluster 0 pgmap v161: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:08 smithi064 bash[25056]: audit 2023-12-07T17:21:07.257350+0000 mgr.y (mgr.24626) 226 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:08 smithi064 bash[36207]: cluster 2023-12-07T17:21:06.648586+0000 mgr.y (mgr.24626) 225 : cluster 0 pgmap v161: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:08.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:08 smithi064 bash[36207]: audit 2023-12-07T17:21:07.257350+0000 mgr.y (mgr.24626) 226 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:10.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:10 smithi184 bash[31336]: cluster 2023-12-07T17:21:08.649595+0000 mgr.y (mgr.24626) 227 : cluster 0 pgmap v162: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:10.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:10 smithi064 bash[25056]: cluster 2023-12-07T17:21:08.649595+0000 mgr.y (mgr.24626) 227 : cluster 0 pgmap v162: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:10.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:10 smithi064 bash[36207]: cluster 2023-12-07T17:21:08.649595+0000 mgr.y (mgr.24626) 227 : cluster 0 pgmap v162: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:12.120 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:12 smithi064 bash[36207]: cluster 2023-12-07T17:21:10.650180+0000 mgr.y (mgr.24626) 228 : cluster 0 pgmap v163: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:12.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:12 smithi064 bash[25056]: cluster 2023-12-07T17:21:10.650180+0000 mgr.y (mgr.24626) 228 : cluster 0 pgmap v163: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:12.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:12 smithi184 bash[31336]: cluster 2023-12-07T17:21:10.650180+0000 mgr.y (mgr.24626) 228 : cluster 0 pgmap v163: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:14.162 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:21:14.162 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:21:14.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:14 smithi184 bash[31336]: cluster 2023-12-07T17:21:12.651045+0000 mgr.y (mgr.24626) 229 : cluster 0 pgmap v164: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:14.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:14 smithi064 bash[36207]: cluster 2023-12-07T17:21:12.651045+0000 mgr.y (mgr.24626) 229 : cluster 0 pgmap v164: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:14.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:14 smithi064 bash[25056]: cluster 2023-12-07T17:21:12.651045+0000 mgr.y (mgr.24626) 229 : cluster 0 pgmap v164: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:15.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:15 smithi184 bash[31336]: audit 2023-12-07T17:21:14.158986+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.64:0/3649100584' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:15.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:15 smithi064 bash[25056]: audit 2023-12-07T17:21:14.158986+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.64:0/3649100584' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:15.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:15 smithi064 bash[36207]: audit 2023-12-07T17:21:14.158986+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.64:0/3649100584' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:16.037 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:16.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:16 smithi184 bash[31336]: cluster 2023-12-07T17:21:14.651722+0000 mgr.y (mgr.24626) 230 : cluster 0 pgmap v165: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:16.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:16 smithi064 bash[25056]: cluster 2023-12-07T17:21:14.651722+0000 mgr.y (mgr.24626) 230 : cluster 0 pgmap v165: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:16.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:16 smithi064 bash[36207]: cluster 2023-12-07T17:21:14.651722+0000 mgr.y (mgr.24626) 230 : cluster 0 pgmap v165: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:17.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:17.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:21:18.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:18 smithi184 bash[31336]: cluster 2023-12-07T17:21:16.652308+0000 mgr.y (mgr.24626) 231 : cluster 0 pgmap v166: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:18.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:18 smithi184 bash[31336]: audit 2023-12-07T17:21:17.266685+0000 mgr.y (mgr.24626) 232 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:18.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:18 smithi064 bash[25056]: cluster 2023-12-07T17:21:16.652308+0000 mgr.y (mgr.24626) 231 : cluster 0 pgmap v166: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:18.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:18 smithi064 bash[25056]: audit 2023-12-07T17:21:17.266685+0000 mgr.y (mgr.24626) 232 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:18.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:18 smithi064 bash[36207]: cluster 2023-12-07T17:21:16.652308+0000 mgr.y (mgr.24626) 231 : cluster 0 pgmap v166: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:18.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:18 smithi064 bash[36207]: audit 2023-12-07T17:21:17.266685+0000 mgr.y (mgr.24626) 232 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:20.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:20 smithi184 bash[31336]: cluster 2023-12-07T17:21:18.653296+0000 mgr.y (mgr.24626) 233 : cluster 0 pgmap v167: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:20.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:20 smithi064 bash[25056]: cluster 2023-12-07T17:21:18.653296+0000 mgr.y (mgr.24626) 233 : cluster 0 pgmap v167: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:20.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:20 smithi064 bash[36207]: cluster 2023-12-07T17:21:18.653296+0000 mgr.y (mgr.24626) 233 : cluster 0 pgmap v167: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:22.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:22 smithi184 bash[31336]: cluster 2023-12-07T17:21:20.654052+0000 mgr.y (mgr.24626) 234 : cluster 0 pgmap v168: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:22.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:22 smithi184 bash[31336]: audit 2023-12-07T17:21:21.523345+0000 mon.c (mon.2) 102 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:22 smithi064 bash[25056]: cluster 2023-12-07T17:21:20.654052+0000 mgr.y (mgr.24626) 234 : cluster 0 pgmap v168: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:22.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:22 smithi064 bash[25056]: audit 2023-12-07T17:21:21.523345+0000 mon.c (mon.2) 102 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:22.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:22 smithi064 bash[36207]: cluster 2023-12-07T17:21:20.654052+0000 mgr.y (mgr.24626) 234 : cluster 0 pgmap v168: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:22.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:22 smithi064 bash[36207]: audit 2023-12-07T17:21:21.523345+0000 mon.c (mon.2) 102 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:22.761 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:24.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:24 smithi184 bash[31336]: cluster 2023-12-07T17:21:22.654930+0000 mgr.y (mgr.24626) 235 : cluster 0 pgmap v169: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:24.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:24 smithi064 bash[25056]: cluster 2023-12-07T17:21:22.654930+0000 mgr.y (mgr.24626) 235 : cluster 0 pgmap v169: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:24.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:24 smithi064 bash[36207]: cluster 2023-12-07T17:21:22.654930+0000 mgr.y (mgr.24626) 235 : cluster 0 pgmap v169: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:25.870 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:26.041 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:21:26.041 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:21:26.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:26 smithi184 bash[31336]: cluster 2023-12-07T17:21:24.655555+0000 mgr.y (mgr.24626) 236 : cluster 0 pgmap v170: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:26.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:26 smithi184 bash[31336]: audit 2023-12-07T17:21:26.038363+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.64:0/2662648671' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:26.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:26 smithi064 bash[36207]: cluster 2023-12-07T17:21:24.655555+0000 mgr.y (mgr.24626) 236 : cluster 0 pgmap v170: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:26.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:26 smithi064 bash[36207]: audit 2023-12-07T17:21:26.038363+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.64:0/2662648671' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:26.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:26 smithi064 bash[25056]: cluster 2023-12-07T17:21:24.655555+0000 mgr.y (mgr.24626) 236 : cluster 0 pgmap v170: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:21:26.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:26 smithi064 bash[25056]: audit 2023-12-07T17:21:26.038363+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.64:0/2662648671' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:27.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:28.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:28 smithi184 bash[31336]: cluster 2023-12-07T17:21:26.656162+0000 mgr.y (mgr.24626) 237 : cluster 0 pgmap v171: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:28.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:28 smithi184 bash[31336]: audit 2023-12-07T17:21:27.278044+0000 mgr.y (mgr.24626) 238 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:28.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:28 smithi064 bash[36207]: cluster 2023-12-07T17:21:26.656162+0000 mgr.y (mgr.24626) 237 : cluster 0 pgmap v171: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:28.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:28 smithi064 bash[36207]: audit 2023-12-07T17:21:27.278044+0000 mgr.y (mgr.24626) 238 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:28 smithi064 bash[25056]: cluster 2023-12-07T17:21:26.656162+0000 mgr.y (mgr.24626) 237 : cluster 0 pgmap v171: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:28 smithi064 bash[25056]: audit 2023-12-07T17:21:27.278044+0000 mgr.y (mgr.24626) 238 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:29.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:21:30.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:30 smithi184 bash[31336]: cluster 2023-12-07T17:21:28.656987+0000 mgr.y (mgr.24626) 239 : cluster 0 pgmap v172: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:30.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:30 smithi064 bash[25056]: cluster 2023-12-07T17:21:28.656987+0000 mgr.y (mgr.24626) 239 : cluster 0 pgmap v172: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:30.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:30 smithi064 bash[36207]: cluster 2023-12-07T17:21:28.656987+0000 mgr.y (mgr.24626) 239 : cluster 0 pgmap v172: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:32.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:32 smithi064 bash[25056]: cluster 2023-12-07T17:21:30.657634+0000 mgr.y (mgr.24626) 240 : cluster 0 pgmap v173: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:32.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:32 smithi064 bash[36207]: cluster 2023-12-07T17:21:30.657634+0000 mgr.y (mgr.24626) 240 : cluster 0 pgmap v173: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:32.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:32 smithi184 bash[31336]: cluster 2023-12-07T17:21:30.657634+0000 mgr.y (mgr.24626) 240 : cluster 0 pgmap v173: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:34.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:34 smithi064 bash[25056]: cluster 2023-12-07T17:21:32.658208+0000 mgr.y (mgr.24626) 241 : cluster 0 pgmap v174: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:34.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:34 smithi064 bash[36207]: cluster 2023-12-07T17:21:32.658208+0000 mgr.y (mgr.24626) 241 : cluster 0 pgmap v174: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:34.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:34 smithi184 bash[31336]: cluster 2023-12-07T17:21:32.658208+0000 mgr.y (mgr.24626) 241 : cluster 0 pgmap v174: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:35.957 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:35 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:36.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:36 smithi064 bash[25056]: cluster 2023-12-07T17:21:34.658842+0000 mgr.y (mgr.24626) 242 : cluster 0 pgmap v175: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:36.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:36 smithi064 bash[36207]: cluster 2023-12-07T17:21:34.658842+0000 mgr.y (mgr.24626) 242 : cluster 0 pgmap v175: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:36.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:36 smithi184 bash[31336]: cluster 2023-12-07T17:21:34.658842+0000 mgr.y (mgr.24626) 242 : cluster 0 pgmap v175: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:37.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:37 smithi064 bash[25056]: audit 2023-12-07T17:21:36.546330+0000 mon.c (mon.2) 103 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:37.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:37 smithi064 bash[36207]: audit 2023-12-07T17:21:36.546330+0000 mon.c (mon.2) 103 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:37.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:37.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:37 smithi184 bash[31336]: audit 2023-12-07T17:21:36.546330+0000 mon.c (mon.2) 103 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:38.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:21:38.113 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:21:38.340 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:38 smithi064 bash[25056]: cluster 2023-12-07T17:21:36.659369+0000 mgr.y (mgr.24626) 243 : cluster 0 pgmap v176: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:38.340 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:38 smithi064 bash[36207]: cluster 2023-12-07T17:21:36.659369+0000 mgr.y (mgr.24626) 243 : cluster 0 pgmap v176: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:38.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:38 smithi184 bash[31336]: cluster 2023-12-07T17:21:36.659369+0000 mgr.y (mgr.24626) 243 : cluster 0 pgmap v176: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:39.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:39 smithi064 bash[25056]: audit 2023-12-07T17:21:37.289293+0000 mgr.y (mgr.24626) 244 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:39.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:39 smithi064 bash[25056]: audit 2023-12-07T17:21:38.110127+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.64:0/1115970350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:39.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:39 smithi064 bash[36207]: audit 2023-12-07T17:21:37.289293+0000 mgr.y (mgr.24626) 244 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:39.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:39 smithi064 bash[36207]: audit 2023-12-07T17:21:38.110127+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.64:0/1115970350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:39.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:39 smithi184 bash[31336]: audit 2023-12-07T17:21:37.289293+0000 mgr.y (mgr.24626) 244 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:39.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:39 smithi184 bash[31336]: audit 2023-12-07T17:21:38.110127+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.64:0/1115970350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:40.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:40 smithi064 bash[25056]: cluster 2023-12-07T17:21:38.660293+0000 mgr.y (mgr.24626) 245 : cluster 0 pgmap v177: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:40.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:40 smithi064 bash[36207]: cluster 2023-12-07T17:21:38.660293+0000 mgr.y (mgr.24626) 245 : cluster 0 pgmap v177: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:40.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:40 smithi184 bash[31336]: cluster 2023-12-07T17:21:38.660293+0000 mgr.y (mgr.24626) 245 : cluster 0 pgmap v177: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:41.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:21:42.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:42 smithi064 bash[25056]: cluster 2023-12-07T17:21:40.660915+0000 mgr.y (mgr.24626) 246 : cluster 0 pgmap v178: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:42.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:42 smithi064 bash[36207]: cluster 2023-12-07T17:21:40.660915+0000 mgr.y (mgr.24626) 246 : cluster 0 pgmap v178: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:42.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:42 smithi184 bash[31336]: cluster 2023-12-07T17:21:40.660915+0000 mgr.y (mgr.24626) 246 : cluster 0 pgmap v178: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:43.370 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:43 smithi064 bash[36207]: audit 2023-12-07T17:21:42.731737+0000 mon.c (mon.2) 105 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:21:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:43 smithi064 bash[25056]: audit 2023-12-07T17:21:42.731737+0000 mon.c (mon.2) 105 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:21:43.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:43 smithi184 bash[31336]: audit 2023-12-07T17:21:42.731737+0000 mon.c (mon.2) 105 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[25056]: cluster 2023-12-07T17:21:42.661898+0000 mgr.y (mgr.24626) 247 : cluster 0 pgmap v179: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[25056]: audit 2023-12-07T17:21:43.472711+0000 mon.c (mon.2) 106 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[25056]: audit 2023-12-07T17:21:43.474781+0000 mon.c (mon.2) 107 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[25056]: audit 2023-12-07T17:21:43.485463+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[36207]: cluster 2023-12-07T17:21:42.661898+0000 mgr.y (mgr.24626) 247 : cluster 0 pgmap v179: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[36207]: audit 2023-12-07T17:21:43.472711+0000 mon.c (mon.2) 106 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:21:44.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[36207]: audit 2023-12-07T17:21:43.474781+0000 mon.c (mon.2) 107 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:21:44.372 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:44 smithi064 bash[36207]: audit 2023-12-07T17:21:43.485463+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:21:44.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:44 smithi184 bash[31336]: cluster 2023-12-07T17:21:42.661898+0000 mgr.y (mgr.24626) 247 : cluster 0 pgmap v179: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:44.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:44 smithi184 bash[31336]: audit 2023-12-07T17:21:43.472711+0000 mon.c (mon.2) 106 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:21:44.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:44 smithi184 bash[31336]: audit 2023-12-07T17:21:43.474781+0000 mon.c (mon.2) 107 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:21:44.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:44 smithi184 bash[31336]: audit 2023-12-07T17:21:43.485463+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:21:45.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:45 smithi184 bash[31336]: cluster 2023-12-07T17:21:43.478111+0000 mgr.y (mgr.24626) 248 : cluster 0 pgmap v180: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2023-12-07T17:21:45.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:45 smithi184 bash[31336]: cluster 2023-12-07T17:21:43.478592+0000 mgr.y (mgr.24626) 249 : cluster 0 pgmap v181: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:45.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:45 smithi064 bash[25056]: cluster 2023-12-07T17:21:43.478111+0000 mgr.y (mgr.24626) 248 : cluster 0 pgmap v180: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2023-12-07T17:21:45.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:45 smithi064 bash[25056]: cluster 2023-12-07T17:21:43.478592+0000 mgr.y (mgr.24626) 249 : cluster 0 pgmap v181: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:45.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:45 smithi064 bash[36207]: cluster 2023-12-07T17:21:43.478111+0000 mgr.y (mgr.24626) 248 : cluster 0 pgmap v180: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2023-12-07T17:21:45.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:45 smithi064 bash[36207]: cluster 2023-12-07T17:21:43.478592+0000 mgr.y (mgr.24626) 249 : cluster 0 pgmap v181: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:46.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:46.724 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:47.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:47.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:47 smithi184 bash[31336]: cluster 2023-12-07T17:21:45.479230+0000 mgr.y (mgr.24626) 250 : cluster 0 pgmap v182: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:47.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:47 smithi064 bash[36207]: cluster 2023-12-07T17:21:45.479230+0000 mgr.y (mgr.24626) 250 : cluster 0 pgmap v182: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:47.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:47 smithi064 bash[25056]: cluster 2023-12-07T17:21:45.479230+0000 mgr.y (mgr.24626) 250 : cluster 0 pgmap v182: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:49.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:49 smithi184 bash[31336]: audit 2023-12-07T17:21:47.298535+0000 mgr.y (mgr.24626) 251 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:49.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:49 smithi184 bash[31336]: cluster 2023-12-07T17:21:47.480186+0000 mgr.y (mgr.24626) 252 : cluster 0 pgmap v183: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:49.593 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:49 smithi064 bash[25056]: audit 2023-12-07T17:21:47.298535+0000 mgr.y (mgr.24626) 251 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:49.593 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:49 smithi064 bash[25056]: cluster 2023-12-07T17:21:47.480186+0000 mgr.y (mgr.24626) 252 : cluster 0 pgmap v183: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:49.594 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:49 smithi064 bash[36207]: audit 2023-12-07T17:21:47.298535+0000 mgr.y (mgr.24626) 251 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:49.594 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:49 smithi064 bash[36207]: cluster 2023-12-07T17:21:47.480186+0000 mgr.y (mgr.24626) 252 : cluster 0 pgmap v183: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:50.080 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:21:50.080 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:21:50.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:50 smithi184 bash[31336]: audit 2023-12-07T17:21:50.076921+0000 mon.c (mon.2) 108 : audit 0 from='client.? 172.21.15.64:0/3282752875' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:50.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:50 smithi064 bash[36207]: audit 2023-12-07T17:21:50.076921+0000 mon.c (mon.2) 108 : audit 0 from='client.? 172.21.15.64:0/3282752875' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:50.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:50 smithi064 bash[25056]: audit 2023-12-07T17:21:50.076921+0000 mon.c (mon.2) 108 : audit 0 from='client.? 172.21.15.64:0/3282752875' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:21:51.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:51 smithi184 bash[31336]: cluster 2023-12-07T17:21:49.480653+0000 mgr.y (mgr.24626) 253 : cluster 0 pgmap v184: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:51.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:51 smithi064 bash[25056]: cluster 2023-12-07T17:21:49.480653+0000 mgr.y (mgr.24626) 253 : cluster 0 pgmap v184: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:51.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:51 smithi064 bash[36207]: cluster 2023-12-07T17:21:49.480653+0000 mgr.y (mgr.24626) 253 : cluster 0 pgmap v184: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:21:52.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:52 smithi184 bash[31336]: audit 2023-12-07T17:21:51.564495+0000 mon.c (mon.2) 109 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:52.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:52 smithi064 bash[36207]: audit 2023-12-07T17:21:51.564495+0000 mon.c (mon.2) 109 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:52.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:52 smithi064 bash[25056]: audit 2023-12-07T17:21:51.564495+0000 mon.c (mon.2) 109 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:21:53.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:53 smithi184 bash[31336]: cluster 2023-12-07T17:21:51.481251+0000 mgr.y (mgr.24626) 254 : cluster 0 pgmap v185: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 580 B/s rd, 0 op/s 2023-12-07T17:21:53.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:53 smithi064 bash[36207]: cluster 2023-12-07T17:21:51.481251+0000 mgr.y (mgr.24626) 254 : cluster 0 pgmap v185: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 580 B/s rd, 0 op/s 2023-12-07T17:21:53.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:53 smithi064 bash[25056]: cluster 2023-12-07T17:21:51.481251+0000 mgr.y (mgr.24626) 254 : cluster 0 pgmap v185: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 580 B/s rd, 0 op/s 2023-12-07T17:21:53.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:21:55.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:55 smithi184 bash[31336]: cluster 2023-12-07T17:21:53.482176+0000 mgr.y (mgr.24626) 255 : cluster 0 pgmap v186: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:21:55.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:55 smithi064 bash[25056]: cluster 2023-12-07T17:21:53.482176+0000 mgr.y (mgr.24626) 255 : cluster 0 pgmap v186: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:21:55.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:55 smithi064 bash[36207]: cluster 2023-12-07T17:21:53.482176+0000 mgr.y (mgr.24626) 255 : cluster 0 pgmap v186: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:21:56.120 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:21:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:21:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:21:57.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:21:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:21:57.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:57 smithi184 bash[31336]: cluster 2023-12-07T17:21:55.482844+0000 mgr.y (mgr.24626) 256 : cluster 0 pgmap v187: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:57.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:57 smithi064 bash[25056]: cluster 2023-12-07T17:21:55.482844+0000 mgr.y (mgr.24626) 256 : cluster 0 pgmap v187: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:57.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:57 smithi064 bash[36207]: cluster 2023-12-07T17:21:55.482844+0000 mgr.y (mgr.24626) 256 : cluster 0 pgmap v187: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:21:58.652 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:21:59.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:59 smithi184 bash[31336]: audit 2023-12-07T17:21:57.309781+0000 mgr.y (mgr.24626) 257 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:59.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:21:59 smithi184 bash[31336]: cluster 2023-12-07T17:21:57.483660+0000 mgr.y (mgr.24626) 258 : cluster 0 pgmap v188: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:59.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:59 smithi064 bash[36207]: audit 2023-12-07T17:21:57.309781+0000 mgr.y (mgr.24626) 257 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:59.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:21:59 smithi064 bash[36207]: cluster 2023-12-07T17:21:57.483660+0000 mgr.y (mgr.24626) 258 : cluster 0 pgmap v188: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:21:59.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:59 smithi064 bash[25056]: audit 2023-12-07T17:21:57.309781+0000 mgr.y (mgr.24626) 257 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:21:59.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:21:59 smithi064 bash[25056]: cluster 2023-12-07T17:21:57.483660+0000 mgr.y (mgr.24626) 258 : cluster 0 pgmap v188: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:01.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:01 smithi184 bash[31336]: cluster 2023-12-07T17:21:59.484096+0000 mgr.y (mgr.24626) 259 : cluster 0 pgmap v189: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:01.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:01 smithi064 bash[25056]: cluster 2023-12-07T17:21:59.484096+0000 mgr.y (mgr.24626) 259 : cluster 0 pgmap v189: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:01.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:01 smithi064 bash[36207]: cluster 2023-12-07T17:21:59.484096+0000 mgr.y (mgr.24626) 259 : cluster 0 pgmap v189: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:02.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:22:02.181 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:22:03.373 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:03 smithi064 bash[36207]: cluster 2023-12-07T17:22:01.484758+0000 mgr.y (mgr.24626) 260 : cluster 0 pgmap v190: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:03.374 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:03 smithi064 bash[36207]: audit 2023-12-07T17:22:02.177889+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.64:0/3227931449' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:03.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:03 smithi064 bash[25056]: cluster 2023-12-07T17:22:01.484758+0000 mgr.y (mgr.24626) 260 : cluster 0 pgmap v190: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:03.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:03 smithi064 bash[25056]: audit 2023-12-07T17:22:02.177889+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.64:0/3227931449' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:03.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:03 smithi184 bash[31336]: cluster 2023-12-07T17:22:01.484758+0000 mgr.y (mgr.24626) 260 : cluster 0 pgmap v190: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:03.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:03 smithi184 bash[31336]: audit 2023-12-07T17:22:02.177889+0000 mon.a (mon.0) 831 : audit 0 from='client.? 172.21.15.64:0/3227931449' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:05.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:05 smithi184 bash[31336]: cluster 2023-12-07T17:22:03.485671+0000 mgr.y (mgr.24626) 261 : cluster 0 pgmap v191: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:05.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:05 smithi064 bash[25056]: cluster 2023-12-07T17:22:03.485671+0000 mgr.y (mgr.24626) 261 : cluster 0 pgmap v191: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:05.622 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:05 smithi064 bash[36207]: cluster 2023-12-07T17:22:03.485671+0000 mgr.y (mgr.24626) 261 : cluster 0 pgmap v191: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:05.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:22:05.951 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:07.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:07.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:07 smithi184 bash[31336]: cluster 2023-12-07T17:22:05.486344+0000 mgr.y (mgr.24626) 262 : cluster 0 pgmap v192: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:07.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:07 smithi184 bash[31336]: audit 2023-12-07T17:22:06.583437+0000 mon.c (mon.2) 110 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:07.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:07 smithi064 bash[36207]: cluster 2023-12-07T17:22:05.486344+0000 mgr.y (mgr.24626) 262 : cluster 0 pgmap v192: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:07.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:07 smithi064 bash[36207]: audit 2023-12-07T17:22:06.583437+0000 mon.c (mon.2) 110 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:07.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:07 smithi064 bash[25056]: cluster 2023-12-07T17:22:05.486344+0000 mgr.y (mgr.24626) 262 : cluster 0 pgmap v192: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:07.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:07 smithi064 bash[25056]: audit 2023-12-07T17:22:06.583437+0000 mon.c (mon.2) 110 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:09.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:09 smithi184 bash[31336]: audit 2023-12-07T17:22:07.321020+0000 mgr.y (mgr.24626) 263 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:09.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:09 smithi184 bash[31336]: cluster 2023-12-07T17:22:07.487126+0000 mgr.y (mgr.24626) 264 : cluster 0 pgmap v193: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:09.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:09 smithi064 bash[25056]: audit 2023-12-07T17:22:07.321020+0000 mgr.y (mgr.24626) 263 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:09.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:09 smithi064 bash[25056]: cluster 2023-12-07T17:22:07.487126+0000 mgr.y (mgr.24626) 264 : cluster 0 pgmap v193: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:09.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:09 smithi064 bash[36207]: audit 2023-12-07T17:22:07.321020+0000 mgr.y (mgr.24626) 263 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:09.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:09 smithi064 bash[36207]: cluster 2023-12-07T17:22:07.487126+0000 mgr.y (mgr.24626) 264 : cluster 0 pgmap v193: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:10.722 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:22:11.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:11 smithi184 bash[31336]: cluster 2023-12-07T17:22:09.487760+0000 mgr.y (mgr.24626) 265 : cluster 0 pgmap v194: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:11.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:11 smithi064 bash[25056]: cluster 2023-12-07T17:22:09.487760+0000 mgr.y (mgr.24626) 265 : cluster 0 pgmap v194: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:11.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:11 smithi064 bash[36207]: cluster 2023-12-07T17:22:09.487760+0000 mgr.y (mgr.24626) 265 : cluster 0 pgmap v194: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:13.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:13 smithi184 bash[31336]: cluster 2023-12-07T17:22:11.488347+0000 mgr.y (mgr.24626) 266 : cluster 0 pgmap v195: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:13.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:13 smithi064 bash[25056]: cluster 2023-12-07T17:22:11.488347+0000 mgr.y (mgr.24626) 266 : cluster 0 pgmap v195: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:13.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:13 smithi064 bash[36207]: cluster 2023-12-07T17:22:11.488347+0000 mgr.y (mgr.24626) 266 : cluster 0 pgmap v195: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:22:14.112 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:22:14.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:14 smithi184 bash[31336]: audit 2023-12-07T17:22:14.108531+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.64:0/480608725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:14.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:14 smithi064 bash[36207]: audit 2023-12-07T17:22:14.108531+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.64:0/480608725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:14.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:14 smithi064 bash[25056]: audit 2023-12-07T17:22:14.108531+0000 mon.a (mon.0) 832 : audit 0 from='client.? 172.21.15.64:0/480608725' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:15.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:15 smithi064 bash[25056]: cluster 2023-12-07T17:22:13.489326+0000 mgr.y (mgr.24626) 267 : cluster 0 pgmap v196: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:15.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:15 smithi064 bash[36207]: cluster 2023-12-07T17:22:13.489326+0000 mgr.y (mgr.24626) 267 : cluster 0 pgmap v196: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:15.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:15 smithi184 bash[31336]: cluster 2023-12-07T17:22:13.489326+0000 mgr.y (mgr.24626) 267 : cluster 0 pgmap v196: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:16.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:17.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:17 smithi184 bash[31336]: cluster 2023-12-07T17:22:15.490112+0000 mgr.y (mgr.24626) 268 : cluster 0 pgmap v197: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:17.568 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:17.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:17 smithi064 bash[25056]: cluster 2023-12-07T17:22:15.490112+0000 mgr.y (mgr.24626) 268 : cluster 0 pgmap v197: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:17.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:17 smithi064 bash[36207]: cluster 2023-12-07T17:22:15.490112+0000 mgr.y (mgr.24626) 268 : cluster 0 pgmap v197: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:17.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:22:19.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:19 smithi184 bash[31336]: audit 2023-12-07T17:22:17.330427+0000 mgr.y (mgr.24626) 269 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:19.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:19 smithi184 bash[31336]: cluster 2023-12-07T17:22:17.491145+0000 mgr.y (mgr.24626) 270 : cluster 0 pgmap v198: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:19.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:19 smithi064 bash[25056]: audit 2023-12-07T17:22:17.330427+0000 mgr.y (mgr.24626) 269 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:19.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:19 smithi064 bash[25056]: cluster 2023-12-07T17:22:17.491145+0000 mgr.y (mgr.24626) 270 : cluster 0 pgmap v198: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:19.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:19 smithi064 bash[36207]: audit 2023-12-07T17:22:17.330427+0000 mgr.y (mgr.24626) 269 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:19.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:19 smithi064 bash[36207]: cluster 2023-12-07T17:22:17.491145+0000 mgr.y (mgr.24626) 270 : cluster 0 pgmap v198: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:21.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:21 smithi184 bash[31336]: cluster 2023-12-07T17:22:19.491866+0000 mgr.y (mgr.24626) 271 : cluster 0 pgmap v199: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:21.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:21 smithi064 bash[36207]: cluster 2023-12-07T17:22:19.491866+0000 mgr.y (mgr.24626) 271 : cluster 0 pgmap v199: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:21.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:21 smithi064 bash[25056]: cluster 2023-12-07T17:22:19.491866+0000 mgr.y (mgr.24626) 271 : cluster 0 pgmap v199: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:22.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:22 smithi184 bash[31336]: audit 2023-12-07T17:22:21.602464+0000 mon.c (mon.2) 111 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:22.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:22 smithi064 bash[25056]: audit 2023-12-07T17:22:21.602464+0000 mon.c (mon.2) 111 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:22.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:22 smithi064 bash[36207]: audit 2023-12-07T17:22:21.602464+0000 mon.c (mon.2) 111 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:22.758 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:22:23.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:23 smithi184 bash[31336]: cluster 2023-12-07T17:22:21.492502+0000 mgr.y (mgr.24626) 272 : cluster 0 pgmap v200: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:23.620 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:23 smithi064 bash[36207]: cluster 2023-12-07T17:22:21.492502+0000 mgr.y (mgr.24626) 272 : cluster 0 pgmap v200: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:23.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:23 smithi064 bash[25056]: cluster 2023-12-07T17:22:21.492502+0000 mgr.y (mgr.24626) 272 : cluster 0 pgmap v200: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:25.568 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:25 smithi184 bash[31336]: cluster 2023-12-07T17:22:23.493252+0000 mgr.y (mgr.24626) 273 : cluster 0 pgmap v201: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:25.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:25 smithi064 bash[36207]: cluster 2023-12-07T17:22:23.493252+0000 mgr.y (mgr.24626) 273 : cluster 0 pgmap v201: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:25.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:25 smithi064 bash[25056]: cluster 2023-12-07T17:22:23.493252+0000 mgr.y (mgr.24626) 273 : cluster 0 pgmap v201: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:26.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:26.206 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:22:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:22:27.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:26 smithi064 bash[25056]: cluster 2023-12-07T17:22:25.493912+0000 mgr.y (mgr.24626) 274 : cluster 0 pgmap v202: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:27.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:26 smithi064 bash[25056]: audit 2023-12-07T17:22:26.202905+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.64:0/2501403970' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:27.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:26 smithi064 bash[36207]: cluster 2023-12-07T17:22:25.493912+0000 mgr.y (mgr.24626) 274 : cluster 0 pgmap v202: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:27.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:26 smithi064 bash[36207]: audit 2023-12-07T17:22:26.202905+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.64:0/2501403970' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:27.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:26 smithi184 bash[31336]: cluster 2023-12-07T17:22:25.493912+0000 mgr.y (mgr.24626) 274 : cluster 0 pgmap v202: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2023-12-07T17:22:27.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:26 smithi184 bash[31336]: audit 2023-12-07T17:22:26.202905+0000 mon.a (mon.0) 833 : audit 0 from='client.? 172.21.15.64:0/2501403970' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:27.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:28 smithi064 bash[25056]: audit 2023-12-07T17:22:27.338424+0000 mgr.y (mgr.24626) 275 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:28 smithi064 bash[25056]: cluster 2023-12-07T17:22:27.494727+0000 mgr.y (mgr.24626) 276 : cluster 0 pgmap v203: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:29.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:28 smithi064 bash[36207]: audit 2023-12-07T17:22:27.338424+0000 mgr.y (mgr.24626) 275 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:29.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:28 smithi064 bash[36207]: cluster 2023-12-07T17:22:27.494727+0000 mgr.y (mgr.24626) 276 : cluster 0 pgmap v203: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:29.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:28 smithi184 bash[31336]: audit 2023-12-07T17:22:27.338424+0000 mgr.y (mgr.24626) 275 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:29.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:28 smithi184 bash[31336]: cluster 2023-12-07T17:22:27.494727+0000 mgr.y (mgr.24626) 276 : cluster 0 pgmap v203: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:30.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:22:31.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:30 smithi064 bash[25056]: cluster 2023-12-07T17:22:29.495315+0000 mgr.y (mgr.24626) 277 : cluster 0 pgmap v204: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:31.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:30 smithi064 bash[36207]: cluster 2023-12-07T17:22:29.495315+0000 mgr.y (mgr.24626) 277 : cluster 0 pgmap v204: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:31.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:30 smithi184 bash[31336]: cluster 2023-12-07T17:22:29.495315+0000 mgr.y (mgr.24626) 277 : cluster 0 pgmap v204: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:33.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:32 smithi064 bash[25056]: cluster 2023-12-07T17:22:31.496023+0000 mgr.y (mgr.24626) 278 : cluster 0 pgmap v205: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:33.124 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:32 smithi064 bash[36207]: cluster 2023-12-07T17:22:31.496023+0000 mgr.y (mgr.24626) 278 : cluster 0 pgmap v205: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:33.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:32 smithi184 bash[31336]: cluster 2023-12-07T17:22:31.496023+0000 mgr.y (mgr.24626) 278 : cluster 0 pgmap v205: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:34.788 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:22:35.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:34 smithi064 bash[25056]: cluster 2023-12-07T17:22:33.496886+0000 mgr.y (mgr.24626) 279 : cluster 0 pgmap v206: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:35.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:34 smithi064 bash[36207]: cluster 2023-12-07T17:22:33.496886+0000 mgr.y (mgr.24626) 279 : cluster 0 pgmap v206: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:35.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:34 smithi184 bash[31336]: cluster 2023-12-07T17:22:33.496886+0000 mgr.y (mgr.24626) 279 : cluster 0 pgmap v206: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:35.952 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:35 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:36 smithi064 bash[25056]: cluster 2023-12-07T17:22:35.497381+0000 mgr.y (mgr.24626) 280 : cluster 0 pgmap v207: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:37.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:36 smithi064 bash[25056]: audit 2023-12-07T17:22:36.620875+0000 mon.c (mon.2) 112 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:37.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:36 smithi064 bash[36207]: cluster 2023-12-07T17:22:35.497381+0000 mgr.y (mgr.24626) 280 : cluster 0 pgmap v207: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:37.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:36 smithi064 bash[36207]: audit 2023-12-07T17:22:36.620875+0000 mon.c (mon.2) 112 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:37.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:36 smithi184 bash[31336]: cluster 2023-12-07T17:22:35.497381+0000 mgr.y (mgr.24626) 280 : cluster 0 pgmap v207: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:37.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:36 smithi184 bash[31336]: audit 2023-12-07T17:22:36.620875+0000 mon.c (mon.2) 112 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:37.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:37 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:38.362 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:22:38.363 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[36207]: audit 2023-12-07T17:22:37.348091+0000 mgr.y (mgr.24626) 281 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[36207]: cluster 2023-12-07T17:22:37.498212+0000 mgr.y (mgr.24626) 282 : cluster 0 pgmap v208: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[36207]: audit 2023-12-07T17:22:38.359489+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.64:0/568526256' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[25056]: audit 2023-12-07T17:22:37.348091+0000 mgr.y (mgr.24626) 281 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[25056]: cluster 2023-12-07T17:22:37.498212+0000 mgr.y (mgr.24626) 282 : cluster 0 pgmap v208: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:39.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:38 smithi064 bash[25056]: audit 2023-12-07T17:22:38.359489+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.64:0/568526256' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:39.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:38 smithi184 bash[31336]: audit 2023-12-07T17:22:37.348091+0000 mgr.y (mgr.24626) 281 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:39.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:38 smithi184 bash[31336]: cluster 2023-12-07T17:22:37.498212+0000 mgr.y (mgr.24626) 282 : cluster 0 pgmap v208: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:39.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:38 smithi184 bash[31336]: audit 2023-12-07T17:22:38.359489+0000 mon.a (mon.0) 834 : audit 0 from='client.? 172.21.15.64:0/568526256' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:41.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:40 smithi184 bash[31336]: cluster 2023-12-07T17:22:39.498852+0000 mgr.y (mgr.24626) 283 : cluster 0 pgmap v209: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:41.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:40 smithi064 bash[25056]: cluster 2023-12-07T17:22:39.498852+0000 mgr.y (mgr.24626) 283 : cluster 0 pgmap v209: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:41.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:40 smithi064 bash[36207]: cluster 2023-12-07T17:22:39.498852+0000 mgr.y (mgr.24626) 283 : cluster 0 pgmap v209: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:42.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:22:43.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:42 smithi184 bash[31336]: cluster 2023-12-07T17:22:41.499469+0000 mgr.y (mgr.24626) 284 : cluster 0 pgmap v210: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:43.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:42 smithi064 bash[25056]: cluster 2023-12-07T17:22:41.499469+0000 mgr.y (mgr.24626) 284 : cluster 0 pgmap v210: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:43.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:42 smithi064 bash[36207]: cluster 2023-12-07T17:22:41.499469+0000 mgr.y (mgr.24626) 284 : cluster 0 pgmap v210: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:44.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:43 smithi184 bash[31336]: audit 2023-12-07T17:22:43.568729+0000 mon.c (mon.2) 113 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:22:44.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:43 smithi064 bash[25056]: audit 2023-12-07T17:22:43.568729+0000 mon.c (mon.2) 113 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:22:44.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:43 smithi064 bash[36207]: audit 2023-12-07T17:22:43.568729+0000 mon.c (mon.2) 113 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T17:22:45.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: cluster 2023-12-07T17:22:43.500361+0000 mgr.y (mgr.24626) 285 : cluster 0 pgmap v211: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:45.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: audit 2023-12-07T17:22:44.245666+0000 mon.c (mon.2) 114 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:22:45.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: audit 2023-12-07T17:22:44.247622+0000 mon.c (mon.2) 115 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:22:45.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: cluster 2023-12-07T17:22:44.250811+0000 mgr.y (mgr.24626) 286 : cluster 0 pgmap v212: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 0 op/s 2023-12-07T17:22:45.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: cluster 2023-12-07T17:22:44.251222+0000 mgr.y (mgr.24626) 287 : cluster 0 pgmap v213: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:45.319 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:44 smithi184 bash[31336]: audit 2023-12-07T17:22:44.258534+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: cluster 2023-12-07T17:22:43.500361+0000 mgr.y (mgr.24626) 285 : cluster 0 pgmap v211: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: audit 2023-12-07T17:22:44.245666+0000 mon.c (mon.2) 114 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: audit 2023-12-07T17:22:44.247622+0000 mon.c (mon.2) 115 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: cluster 2023-12-07T17:22:44.250811+0000 mgr.y (mgr.24626) 286 : cluster 0 pgmap v212: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 0 op/s 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: cluster 2023-12-07T17:22:44.251222+0000 mgr.y (mgr.24626) 287 : cluster 0 pgmap v213: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:45.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[36207]: audit 2023-12-07T17:22:44.258534+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: cluster 2023-12-07T17:22:43.500361+0000 mgr.y (mgr.24626) 285 : cluster 0 pgmap v211: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: audit 2023-12-07T17:22:44.245666+0000 mon.c (mon.2) 114 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: audit 2023-12-07T17:22:44.247622+0000 mon.c (mon.2) 115 : audit 1 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: cluster 2023-12-07T17:22:44.250811+0000 mgr.y (mgr.24626) 286 : cluster 0 pgmap v212: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 0 op/s 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: cluster 2023-12-07T17:22:44.251222+0000 mgr.y (mgr.24626) 287 : cluster 0 pgmap v213: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:45.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:44 smithi064 bash[25056]: audit 2023-12-07T17:22:44.258534+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24626 ' entity='mgr.y' 2023-12-07T17:22:46.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:45 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:46.933 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:22:47.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:46 smithi064 bash[25056]: cluster 2023-12-07T17:22:46.251879+0000 mgr.y (mgr.24626) 288 : cluster 0 pgmap v214: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 584 B/s rd, 0 op/s 2023-12-07T17:22:47.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:46 smithi064 bash[36207]: cluster 2023-12-07T17:22:46.251879+0000 mgr.y (mgr.24626) 288 : cluster 0 pgmap v214: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 584 B/s rd, 0 op/s 2023-12-07T17:22:47.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:46 smithi184 bash[31336]: cluster 2023-12-07T17:22:46.251879+0000 mgr.y (mgr.24626) 288 : cluster 0 pgmap v214: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 584 B/s rd, 0 op/s 2023-12-07T17:22:47.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:47 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:48.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:48 smithi064 bash[25056]: audit 2023-12-07T17:22:47.356687+0000 mgr.y (mgr.24626) 289 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:48.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:48 smithi064 bash[25056]: cluster 2023-12-07T17:22:48.252818+0000 mgr.y (mgr.24626) 290 : cluster 0 pgmap v215: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:48.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:48 smithi064 bash[36207]: audit 2023-12-07T17:22:47.356687+0000 mgr.y (mgr.24626) 289 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:48.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:48 smithi064 bash[36207]: cluster 2023-12-07T17:22:48.252818+0000 mgr.y (mgr.24626) 290 : cluster 0 pgmap v215: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:48.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:48 smithi184 bash[31336]: audit 2023-12-07T17:22:47.356687+0000 mgr.y (mgr.24626) 289 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:48.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:48 smithi184 bash[31336]: cluster 2023-12-07T17:22:48.252818+0000 mgr.y (mgr.24626) 290 : cluster 0 pgmap v215: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:50.365 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:22:50.365 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:22:50.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:50 smithi064 bash[25056]: cluster 2023-12-07T17:22:50.253300+0000 mgr.y (mgr.24626) 291 : cluster 0 pgmap v216: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:50.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:50 smithi064 bash[36207]: cluster 2023-12-07T17:22:50.253300+0000 mgr.y (mgr.24626) 291 : cluster 0 pgmap v216: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:50.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:50 smithi184 bash[31336]: cluster 2023-12-07T17:22:50.253300+0000 mgr.y (mgr.24626) 291 : cluster 0 pgmap v216: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:51.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:51 smithi064 bash[25056]: audit 2023-12-07T17:22:50.361933+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.64:0/463317155' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:51.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:51 smithi064 bash[36207]: audit 2023-12-07T17:22:50.361933+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.64:0/463317155' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:51.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:51 smithi184 bash[31336]: audit 2023-12-07T17:22:50.361933+0000 mon.a (mon.0) 836 : audit 0 from='client.? 172.21.15.64:0/463317155' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:22:52.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:52 smithi064 bash[25056]: audit 2023-12-07T17:22:51.639731+0000 mon.c (mon.2) 116 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:52.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:52 smithi064 bash[25056]: cluster 2023-12-07T17:22:52.254129+0000 mgr.y (mgr.24626) 292 : cluster 0 pgmap v217: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:52.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:52 smithi064 bash[36207]: audit 2023-12-07T17:22:51.639731+0000 mon.c (mon.2) 116 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:52.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:52 smithi064 bash[36207]: cluster 2023-12-07T17:22:52.254129+0000 mgr.y (mgr.24626) 292 : cluster 0 pgmap v217: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:52.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:52 smithi184 bash[31336]: audit 2023-12-07T17:22:51.639731+0000 mon.c (mon.2) 116 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:22:52.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:52 smithi184 bash[31336]: cluster 2023-12-07T17:22:52.254129+0000 mgr.y (mgr.24626) 292 : cluster 0 pgmap v217: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2023-12-07T17:22:54.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:22:54.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:54 smithi064 bash[25056]: cluster 2023-12-07T17:22:54.254889+0000 mgr.y (mgr.24626) 293 : cluster 0 pgmap v218: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:22:54.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:54 smithi064 bash[36207]: cluster 2023-12-07T17:22:54.254889+0000 mgr.y (mgr.24626) 293 : cluster 0 pgmap v218: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:22:54.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:54 smithi184 bash[31336]: cluster 2023-12-07T17:22:54.254889+0000 mgr.y (mgr.24626) 293 : cluster 0 pgmap v218: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2023-12-07T17:22:56.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:22:55 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:22:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:22:57.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:56 smithi064 bash[25056]: cluster 2023-12-07T17:22:56.255753+0000 mgr.y (mgr.24626) 294 : cluster 0 pgmap v219: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:57.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:56 smithi064 bash[36207]: cluster 2023-12-07T17:22:56.255753+0000 mgr.y (mgr.24626) 294 : cluster 0 pgmap v219: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:57.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:56 smithi184 bash[31336]: cluster 2023-12-07T17:22:56.255753+0000 mgr.y (mgr.24626) 294 : cluster 0 pgmap v219: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:22:57.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:22:57 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:22:58.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:58 smithi064 bash[25056]: audit 2023-12-07T17:22:57.364745+0000 mgr.y (mgr.24626) 295 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:58.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:22:58 smithi064 bash[25056]: cluster 2023-12-07T17:22:58.256704+0000 mgr.y (mgr.24626) 296 : cluster 0 pgmap v220: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:58.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:58 smithi064 bash[36207]: audit 2023-12-07T17:22:57.364745+0000 mgr.y (mgr.24626) 295 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:58.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:22:58 smithi064 bash[36207]: cluster 2023-12-07T17:22:58.256704+0000 mgr.y (mgr.24626) 296 : cluster 0 pgmap v220: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:58.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:58 smithi184 bash[31336]: audit 2023-12-07T17:22:57.364745+0000 mgr.y (mgr.24626) 295 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:22:58.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:22:58 smithi184 bash[31336]: cluster 2023-12-07T17:22:58.256704+0000 mgr.y (mgr.24626) 296 : cluster 0 pgmap v220: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:22:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:23:00.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:00 smithi064 bash[36207]: cluster 2023-12-07T17:23:00.257196+0000 mgr.y (mgr.24626) 297 : cluster 0 pgmap v221: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:00.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:00 smithi064 bash[25056]: cluster 2023-12-07T17:23:00.257196+0000 mgr.y (mgr.24626) 297 : cluster 0 pgmap v221: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:00.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:00 smithi184 bash[31336]: cluster 2023-12-07T17:23:00.257196+0000 mgr.y (mgr.24626) 297 : cluster 0 pgmap v221: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:02.336 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:23:02.336 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:23:02.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:02 smithi064 bash[25056]: cluster 2023-12-07T17:23:02.257998+0000 mgr.y (mgr.24626) 298 : cluster 0 pgmap v222: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:02.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:02 smithi064 bash[25056]: audit 2023-12-07T17:23:02.332902+0000 mon.a (mon.0) 837 : audit 0 from='client.? 172.21.15.64:0/3289651552' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:02.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:02 smithi064 bash[36207]: cluster 2023-12-07T17:23:02.257998+0000 mgr.y (mgr.24626) 298 : cluster 0 pgmap v222: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:02.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:02 smithi064 bash[36207]: audit 2023-12-07T17:23:02.332902+0000 mon.a (mon.0) 837 : audit 0 from='client.? 172.21.15.64:0/3289651552' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:02.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:02 smithi184 bash[31336]: cluster 2023-12-07T17:23:02.257998+0000 mgr.y (mgr.24626) 298 : cluster 0 pgmap v222: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:02.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:02 smithi184 bash[31336]: audit 2023-12-07T17:23:02.332902+0000 mon.a (mon.0) 837 : audit 0 from='client.? 172.21.15.64:0/3289651552' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:04.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:04 smithi064 bash[25056]: cluster 2023-12-07T17:23:04.258644+0000 mgr.y (mgr.24626) 299 : cluster 0 pgmap v223: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:04.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:04 smithi064 bash[36207]: cluster 2023-12-07T17:23:04.258644+0000 mgr.y (mgr.24626) 299 : cluster 0 pgmap v223: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:04.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:04 smithi184 bash[31336]: cluster 2023-12-07T17:23:04.258644+0000 mgr.y (mgr.24626) 299 : cluster 0 pgmap v223: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2023-12-07T17:23:06.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:23:06.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:05 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:23:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:23:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:06 smithi064 bash[25056]: cluster 2023-12-07T17:23:06.259251+0000 mgr.y (mgr.24626) 300 : cluster 0 pgmap v224: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:07.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:06 smithi064 bash[25056]: audit 2023-12-07T17:23:06.659426+0000 mon.c (mon.2) 117 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:07.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:06 smithi064 bash[36207]: cluster 2023-12-07T17:23:06.259251+0000 mgr.y (mgr.24626) 300 : cluster 0 pgmap v224: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:07.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:06 smithi064 bash[36207]: audit 2023-12-07T17:23:06.659426+0000 mon.c (mon.2) 117 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:07.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:06 smithi184 bash[31336]: cluster 2023-12-07T17:23:06.259251+0000 mgr.y (mgr.24626) 300 : cluster 0 pgmap v224: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:07.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:06 smithi184 bash[31336]: audit 2023-12-07T17:23:06.659426+0000 mon.c (mon.2) 117 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:07.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:23:07 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:23:08.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:08 smithi064 bash[36207]: audit 2023-12-07T17:23:07.376057+0000 mgr.y (mgr.24626) 301 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:08.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:08 smithi064 bash[36207]: cluster 2023-12-07T17:23:08.260171+0000 mgr.y (mgr.24626) 302 : cluster 0 pgmap v225: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:08.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:08 smithi064 bash[25056]: audit 2023-12-07T17:23:07.376057+0000 mgr.y (mgr.24626) 301 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:08.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:08 smithi064 bash[25056]: cluster 2023-12-07T17:23:08.260171+0000 mgr.y (mgr.24626) 302 : cluster 0 pgmap v225: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:08.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:08 smithi184 bash[31336]: audit 2023-12-07T17:23:07.376057+0000 mgr.y (mgr.24626) 301 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:08.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:08 smithi184 bash[31336]: cluster 2023-12-07T17:23:08.260171+0000 mgr.y (mgr.24626) 302 : cluster 0 pgmap v225: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:10.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:10 smithi064 bash[25056]: cluster 2023-12-07T17:23:10.260831+0000 mgr.y (mgr.24626) 303 : cluster 0 pgmap v226: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:10.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:10 smithi064 bash[36207]: cluster 2023-12-07T17:23:10.260831+0000 mgr.y (mgr.24626) 303 : cluster 0 pgmap v226: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:10.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:10 smithi184 bash[31336]: cluster 2023-12-07T17:23:10.260831+0000 mgr.y (mgr.24626) 303 : cluster 0 pgmap v226: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:10.885 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:23:12.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:12 smithi064 bash[36207]: cluster 2023-12-07T17:23:12.261762+0000 mgr.y (mgr.24626) 304 : cluster 0 pgmap v227: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:12.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:12 smithi064 bash[25056]: cluster 2023-12-07T17:23:12.261762+0000 mgr.y (mgr.24626) 304 : cluster 0 pgmap v227: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:12.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:12 smithi184 bash[31336]: cluster 2023-12-07T17:23:12.261762+0000 mgr.y (mgr.24626) 304 : cluster 0 pgmap v227: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:14.267 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:23:14.267 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:23:14.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:14 smithi064 bash[25056]: cluster 2023-12-07T17:23:14.262209+0000 mgr.y (mgr.24626) 305 : cluster 0 pgmap v228: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:14.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:14 smithi064 bash[25056]: audit 2023-12-07T17:23:14.263584+0000 mon.c (mon.2) 118 : audit 0 from='client.? 172.21.15.64:0/2964687040' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:14 smithi064 bash[36207]: cluster 2023-12-07T17:23:14.262209+0000 mgr.y (mgr.24626) 305 : cluster 0 pgmap v228: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:14.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:14 smithi064 bash[36207]: audit 2023-12-07T17:23:14.263584+0000 mon.c (mon.2) 118 : audit 0 from='client.? 172.21.15.64:0/2964687040' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:14.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:14 smithi184 bash[31336]: cluster 2023-12-07T17:23:14.262209+0000 mgr.y (mgr.24626) 305 : cluster 0 pgmap v228: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:14.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:14 smithi184 bash[31336]: audit 2023-12-07T17:23:14.263584+0000 mon.c (mon.2) 118 : audit 0 from='client.? 172.21.15.64:0/2964687040' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:16.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:15 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:23:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:23:17.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:16 smithi064 bash[25056]: cluster 2023-12-07T17:23:16.262983+0000 mgr.y (mgr.24626) 306 : cluster 0 pgmap v229: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:17.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:16 smithi064 bash[36207]: cluster 2023-12-07T17:23:16.262983+0000 mgr.y (mgr.24626) 306 : cluster 0 pgmap v229: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:17.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:16 smithi184 bash[31336]: cluster 2023-12-07T17:23:16.262983+0000 mgr.y (mgr.24626) 306 : cluster 0 pgmap v229: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:17.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:23:17 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:23:18.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e068ebcf7ecc6503f24666fb6b152034d3fe1067 shell --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T17:23:18.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:18 smithi064 bash[25056]: audit 2023-12-07T17:23:17.387414+0000 mgr.y (mgr.24626) 307 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:18.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:18 smithi064 bash[25056]: cluster 2023-12-07T17:23:18.263863+0000 mgr.y (mgr.24626) 308 : cluster 0 pgmap v230: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:18.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:18 smithi064 bash[36207]: audit 2023-12-07T17:23:17.387414+0000 mgr.y (mgr.24626) 307 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:18.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:18 smithi064 bash[36207]: cluster 2023-12-07T17:23:18.263863+0000 mgr.y (mgr.24626) 308 : cluster 0 pgmap v230: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:18.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:18 smithi184 bash[31336]: audit 2023-12-07T17:23:17.387414+0000 mgr.y (mgr.24626) 307 : audit 0 from='client.24565 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2023-12-07T17:23:18.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:18 smithi184 bash[31336]: cluster 2023-12-07T17:23:18.263863+0000 mgr.y (mgr.24626) 308 : cluster 0 pgmap v230: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:20.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:20 smithi064 bash[25056]: cluster 2023-12-07T17:23:20.264498+0000 mgr.y (mgr.24626) 309 : cluster 0 pgmap v231: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:20.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:20 smithi064 bash[36207]: cluster 2023-12-07T17:23:20.264498+0000 mgr.y (mgr.24626) 309 : cluster 0 pgmap v231: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:20.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:20 smithi184 bash[31336]: cluster 2023-12-07T17:23:20.264498+0000 mgr.y (mgr.24626) 309 : cluster 0 pgmap v231: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:22.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:21 smithi064 bash[25056]: audit 2023-12-07T17:23:21.678734+0000 mon.c (mon.2) 119 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:22.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:21 smithi064 bash[36207]: audit 2023-12-07T17:23:21.678734+0000 mon.c (mon.2) 119 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:22.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:21 smithi184 bash[31336]: audit 2023-12-07T17:23:21.678734+0000 mon.c (mon.2) 119 : audit 0 from='mgr.24626 172.21.15.64:0/1281697167' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-12-07T17:23:22.841 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T17:23:23.318 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:22 smithi184 bash[31336]: cluster 2023-12-07T17:23:22.265365+0000 mgr.y (mgr.24626) 310 : cluster 0 pgmap v232: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:23.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:22 smithi064 bash[25056]: cluster 2023-12-07T17:23:22.265365+0000 mgr.y (mgr.24626) 310 : cluster 0 pgmap v232: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:23.371 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:22 smithi064 bash[36207]: cluster 2023-12-07T17:23:22.265365+0000 mgr.y (mgr.24626) 310 : cluster 0 pgmap v232: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2023-12-07T17:23:24.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:24 smithi064 bash[25056]: cluster 2023-12-07T17:23:24.266119+0000 mgr.y (mgr.24626) 311 : cluster 0 pgmap v233: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:24.621 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:24 smithi064 bash[36207]: cluster 2023-12-07T17:23:24.266119+0000 mgr.y (mgr.24626) 311 : cluster 0 pgmap v233: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:24.818 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:24 smithi184 bash[31336]: cluster 2023-12-07T17:23:24.266119+0000 mgr.y (mgr.24626) 311 : cluster 0 pgmap v233: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:26.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:25 smithi064 bash[25333]: ::ffff:172.21.15.184 - - [07/Dec/2023:17:23:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2023-12-07T17:23:26.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-12-07T17:23:26.338 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_FAILED_DAEMON":{"severity":"HEALTH_WARN","summary":{"message":"1 failed cephadm daemon(s)","count":1},"muted":false}},"mutes":[]} 2023-12-07T17:23:27.106 INFO:tasks.cephadm:Teardown begin 2023-12-07T17:23:27.106 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/cephadm.py", line 1791, in task healthy(ctx=ctx, config=config) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph.py", line 1478, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph_manager.py", line 3185, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-12-07T17:23:27.110 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:23:27.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:26 smithi064 bash[25056]: cluster 2023-12-07T17:23:26.266747+0000 mgr.y (mgr.24626) 312 : cluster 0 pgmap v234: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:27.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:26 smithi064 bash[25056]: audit 2023-12-07T17:23:26.334572+0000 mon.c (mon.2) 120 : audit 0 from='client.? 172.21.15.64:0/1049303869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:27.121 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:26 smithi064 bash[36207]: cluster 2023-12-07T17:23:26.266747+0000 mgr.y (mgr.24626) 312 : cluster 0 pgmap v234: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:27.122 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:26 smithi064 bash[36207]: audit 2023-12-07T17:23:26.334572+0000 mon.c (mon.2) 120 : audit 0 from='client.? 172.21.15.64:0/1049303869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:27.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:23:27.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:26 smithi184 bash[31336]: cluster 2023-12-07T17:23:26.266747+0000 mgr.y (mgr.24626) 312 : cluster 0 pgmap v234: 36 pgs: 36 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2023-12-07T17:23:27.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:26 smithi184 bash[31336]: audit 2023-12-07T17:23:26.334572+0000 mon.c (mon.2) 120 : audit 0 from='client.? 172.21.15.64:0/1049303869' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T17:23:27.143 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-07T17:23:27.144 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-07T17:23:27.177 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-07T17:23:27.191 INFO:tasks.cephadm:Stopping all daemons... 2023-12-07T17:23:27.191 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-07T17:23:27.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a 2023-12-07T17:23:27.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:27 smithi064 systemd[1]: Stopping Ceph mon.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:27.675 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:27 smithi064 bash[25056]: debug 2023-12-07T17:23:27.390+0000 7f1b0f147700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-07T17:23:27.676 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:27 smithi064 bash[25056]: debug 2023-12-07T17:23:27.390+0000 7f1b0f147700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-07T17:23:27.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:23:27 smithi184 bash[79431]: debug there is no tcmu-runner data available 2023-12-07T17:23:28.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service' 2023-12-07T17:23:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[105613]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-mon-a 2023-12-07T17:23:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[105678]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-mon-a 2023-12-07T17:23:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:28 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-12-07T17:23:28.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Dec 07 17:23:28 smithi064 systemd[1]: Stopped Ceph mon.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:23:28.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:28.377 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-07T17:23:28.377 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2023-12-07T17:23:28.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.c 2023-12-07T17:23:28.694 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE Bus STOPPING 2023-12-07T17:23:28.694 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2023-12-07T17:23:28.694 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE Bus STOPPED 2023-12-07T17:23:28.694 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE Bus STARTING 2023-12-07T17:23:28.694 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:28 smithi064 systemd[1]: Stopping Ceph mon.c for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:28.695 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[36207]: debug 2023-12-07T17:23:28.586+0000 7fabd4338700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-07T17:23:28.695 INFO:journalctl@ceph.mon.c.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[36207]: debug 2023-12-07T17:23:28.586+0000 7fabd4338700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2023-12-07T17:23:29.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE Serving on http://:::9283 2023-12-07T17:23:29.121 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:28 smithi064 bash[25333]: [07/Dec/2023:17:23:28] ENGINE Bus STARTED 2023-12-07T17:23:29.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.c.service' 2023-12-07T17:23:29.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:29.482 INFO:tasks.cephadm.mon.b:Stopped mon.c 2023-12-07T17:23:29.482 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2023-12-07T17:23:29.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.b 2023-12-07T17:23:29.816 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:29 smithi184 systemd[1]: Stopping Ceph mon.b for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:29.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:29 smithi184 bash[31336]: debug 2023-12-07T17:23:29.640+0000 7f492d531700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-07T17:23:29.817 INFO:journalctl@ceph.mon.b.smithi184.stdout:Dec 07 17:23:29 smithi184 bash[31336]: debug 2023-12-07T17:23:29.640+0000 7f492d531700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2023-12-07T17:23:30.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mon.b.service' 2023-12-07T17:23:30.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:30.653 INFO:tasks.cephadm.mon.b:Stopped mon.b 2023-12-07T17:23:30.653 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-07T17:23:30.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y 2023-12-07T17:23:30.921 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:30 smithi064 systemd[1]: Stopping Ceph mgr.y for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:30.921 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:30 smithi064 bash[25333]: debug 2023-12-07T17:23:30.830+0000 7fa230459700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:23:30.922 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:30 smithi064 bash[25333]: debug 2023-12-07T17:23:30.830+0000 7fa230459700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-07T17:23:31.673 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Dec 07 17:23:31 smithi064 bash[105833]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-mgr-y 2023-12-07T17:23:31.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.y.service' 2023-12-07T17:23:31.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:31.710 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-07T17:23:31.710 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-07T17:23:31.710 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.x 2023-12-07T17:23:31.979 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Dec 07 17:23:31 smithi184 systemd[1]: Stopping Ceph mgr.x for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:32.687 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@mgr.x.service' 2023-12-07T17:23:32.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:32.718 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-07T17:23:32.718 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-07T17:23:32.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.0 2023-12-07T17:23:33.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:32 smithi064 systemd[1]: Stopping Ceph osd.0 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:33.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:32 smithi064 bash[39451]: debug 2023-12-07T17:23:32.902+0000 7f4ac3fda700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:23:33.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:32 smithi064 bash[39451]: debug 2023-12-07T17:23:32.902+0000 7f4ac3fda700 -1 osd.0 71 *** Got signal Terminated *** 2023-12-07T17:23:33.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:32 smithi064 bash[39451]: debug 2023-12-07T17:23:32.902+0000 7f4ac3fda700 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:23:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:23:33 smithi184 bash[83299]: ts=2023-12-07T17:23:33.890Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:23:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:23:33 smithi184 bash[83299]: ts=2023-12-07T17:23:33.894Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:23:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:23:33 smithi184 bash[83299]: ts=2023-12-07T17:23:33.894Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:23:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:23:33 smithi184 bash[83299]: ts=2023-12-07T17:23:33.896Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:23:39.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:38 smithi064 bash[105932]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-0 2023-12-07T17:23:39.121 INFO:journalctl@ceph.osd.0.smithi064.stdout:Dec 07 17:23:39 smithi064 bash[105998]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-0 2023-12-07T17:23:41.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.0.service' 2023-12-07T17:23:41.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:41.250 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-07T17:23:41.250 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-07T17:23:41.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.1 2023-12-07T17:23:41.452 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:41 smithi064 systemd[1]: Stopping Ceph osd.1 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:41.871 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:41 smithi064 bash[50070]: debug 2023-12-07T17:23:41.446+0000 7fe92e854700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:23:41.871 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:41 smithi064 bash[50070]: debug 2023-12-07T17:23:41.446+0000 7fe92e854700 -1 osd.1 71 *** Got signal Terminated *** 2023-12-07T17:23:41.871 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:41 smithi064 bash[50070]: debug 2023-12-07T17:23:41.446+0000 7fe92e854700 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:23:47.756 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:47 smithi064 bash[106140]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-1 2023-12-07T17:23:47.756 INFO:journalctl@ceph.osd.1.smithi064.stdout:Dec 07 17:23:47 smithi064 bash[106205]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-1 2023-12-07T17:23:49.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.1.service' 2023-12-07T17:23:49.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:49.727 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-07T17:23:49.727 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-07T17:23:49.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.2 2023-12-07T17:23:49.960 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:49 smithi064 systemd[1]: Stopping Ceph osd.2 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:50.371 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:49 smithi064 bash[61680]: debug 2023-12-07T17:23:49.953+0000 7ff5d0561700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:23:50.371 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:49 smithi064 bash[61680]: debug 2023-12-07T17:23:49.953+0000 7ff5d0561700 -1 osd.2 71 *** Got signal Terminated *** 2023-12-07T17:23:50.371 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:49 smithi064 bash[61680]: debug 2023-12-07T17:23:49.953+0000 7ff5d0561700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:23:56.232 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:55 smithi064 bash[106345]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-2 2023-12-07T17:23:56.232 INFO:journalctl@ceph.osd.2.smithi064.stdout:Dec 07 17:23:56 smithi064 bash[106412]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-2 2023-12-07T17:23:58.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.2.service' 2023-12-07T17:23:58.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:23:58.150 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-07T17:23:58.150 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-07T17:23:58.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.3 2023-12-07T17:23:58.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:23:58 smithi064 systemd[1]: Stopping Ceph osd.3 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:23:58.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:23:58 smithi064 bash[73963]: debug 2023-12-07T17:23:58.341+0000 7f0c0839a700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:23:58.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:23:58 smithi064 bash[73963]: debug 2023-12-07T17:23:58.341+0000 7f0c0839a700 -1 osd.3 71 *** Got signal Terminated *** 2023-12-07T17:23:58.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:23:58 smithi064 bash[73963]: debug 2023-12-07T17:23:58.341+0000 7f0c0839a700 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:24:00.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:23:59 smithi184 bash[45837]: debug 2023-12-07T17:23:59.736+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:01.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:00 smithi184 bash[45837]: debug 2023-12-07T17:24:00.752+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:01.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:24:00 smithi064 bash[73963]: debug 2023-12-07T17:24:00.993+0000 7f0c0118e700 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:35.186380+0000 front 2023-12-07T17:23:35.186571+0000 (oldest deadline 2023-12-07T17:24:00.486379+0000) 2023-12-07T17:24:02.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:01 smithi184 bash[45837]: debug 2023-12-07T17:24:01.720+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:02.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:24:01 smithi064 bash[73963]: debug 2023-12-07T17:24:01.989+0000 7f0c0118e700 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:35.186380+0000 front 2023-12-07T17:23:35.186571+0000 (oldest deadline 2023-12-07T17:24:00.486379+0000) 2023-12-07T17:24:02.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:02 smithi184 bash[45837]: debug 2023-12-07T17:24:02.692+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:02.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:02 smithi184 bash[57324]: debug 2023-12-07T17:24:02.552+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:02.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:02 smithi184 bash[35040]: debug 2023-12-07T17:24:02.596+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:03.371 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:24:02 smithi064 bash[73963]: debug 2023-12-07T17:24:02.997+0000 7f0c0118e700 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:35.186380+0000 front 2023-12-07T17:23:35.186571+0000 (oldest deadline 2023-12-07T17:24:00.486379+0000) 2023-12-07T17:24:03.548 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:03 smithi184 bash[67940]: debug 2023-12-07T17:24:03.192+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:03.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:03 smithi184 bash[35040]: debug 2023-12-07T17:24:03.640+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:03.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:03 smithi184 bash[45837]: debug 2023-12-07T17:24:03.656+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:03.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:03 smithi184 bash[57324]: debug 2023-12-07T17:24:03.544+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:04.531 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:04 smithi184 bash[67940]: debug 2023-12-07T17:24:04.196+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:04.589 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:24:04 smithi064 bash[106554]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-3 2023-12-07T17:24:04.589 INFO:journalctl@ceph.osd.3.smithi064.stdout:Dec 07 17:24:04 smithi064 bash[106622]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-3 2023-12-07T17:24:04.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:04 smithi184 bash[45837]: debug 2023-12-07T17:24:04.608+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:04.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:04 smithi184 bash[35040]: debug 2023-12-07T17:24:04.592+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:04.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:04 smithi184 bash[57324]: debug 2023-12-07T17:24:04.524+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:05.484 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:05 smithi184 bash[67940]: debug 2023-12-07T17:24:05.204+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:05.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:05 smithi184 bash[35040]: debug 2023-12-07T17:24:05.592+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:05.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:05 smithi184 bash[57324]: debug 2023-12-07T17:24:05.476+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:05.819 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:05 smithi184 bash[45837]: debug 2023-12-07T17:24:05.648+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:06.498 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[67940]: debug 2023-12-07T17:24:06.180+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:06.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.3.service' 2023-12-07T17:24:06.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:06.565 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-07T17:24:06.565 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-07T17:24:06.566 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.4 2023-12-07T17:24:06.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[45837]: debug 2023-12-07T17:24:06.672+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:06.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[57324]: debug 2023-12-07T17:24:06.492+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:06.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:06 smithi184 systemd[1]: Stopping Ceph osd.4 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:06.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[35040]: debug 2023-12-07T17:24:06.604+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:06.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[35040]: debug 2023-12-07T17:24:06.740+0000 7fec90549700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:24:06.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[35040]: debug 2023-12-07T17:24:06.740+0000 7fec90549700 -1 osd.4 71 *** Got signal Terminated *** 2023-12-07T17:24:06.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:06 smithi184 bash[35040]: debug 2023-12-07T17:24:06.740+0000 7fec90549700 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:24:07.521 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:07 smithi184 bash[67940]: debug 2023-12-07T17:24:07.176+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:07.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:07 smithi184 bash[35040]: debug 2023-12-07T17:24:07.612+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:07.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:07 smithi184 bash[45837]: debug 2023-12-07T17:24:07.700+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:07.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:07 smithi184 bash[57324]: debug 2023-12-07T17:24:07.516+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:07.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:07 smithi184 bash[57324]: debug 2023-12-07T17:24:07.516+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:08.540 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[67940]: debug 2023-12-07T17:24:08.152+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:08.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[35040]: debug 2023-12-07T17:24:08.640+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:08.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[45837]: debug 2023-12-07T17:24:08.664+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:08.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[45837]: debug 2023-12-07T17:24:08.664+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:08.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[57324]: debug 2023-12-07T17:24:08.532+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:08.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:08 smithi184 bash[57324]: debug 2023-12-07T17:24:08.532+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:09.565 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[67940]: debug 2023-12-07T17:24:09.187+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:09.566 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[67940]: debug 2023-12-07T17:24:09.187+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:09.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[57324]: debug 2023-12-07T17:24:09.559+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:09.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[57324]: debug 2023-12-07T17:24:09.559+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:09.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[35040]: debug 2023-12-07T17:24:09.599+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:09.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[35040]: debug 2023-12-07T17:24:09.599+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.009206+0000 front 2023-12-07T17:23:46.009169+0000 (oldest deadline 2023-12-07T17:24:08.908917+0000) 2023-12-07T17:24:09.819 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[45837]: debug 2023-12-07T17:24:09.675+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:09.819 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:09 smithi184 bash[45837]: debug 2023-12-07T17:24:09.675+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:10.531 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[67940]: debug 2023-12-07T17:24:10.207+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:10.531 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[67940]: debug 2023-12-07T17:24:10.207+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:10.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[45837]: debug 2023-12-07T17:24:10.647+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:10.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[45837]: debug 2023-12-07T17:24:10.647+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:10.818 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[35040]: debug 2023-12-07T17:24:10.563+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:10.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[35040]: debug 2023-12-07T17:24:10.563+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.009206+0000 front 2023-12-07T17:23:46.009169+0000 (oldest deadline 2023-12-07T17:24:08.908917+0000) 2023-12-07T17:24:10.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[57324]: debug 2023-12-07T17:24:10.527+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:10.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:10 smithi184 bash[57324]: debug 2023-12-07T17:24:10.527+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:11.538 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[67940]: debug 2023-12-07T17:24:11.223+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:11.538 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[67940]: debug 2023-12-07T17:24:11.223+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:11.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[45837]: debug 2023-12-07T17:24:11.619+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:11.818 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[45837]: debug 2023-12-07T17:24:11.619+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:11.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[35040]: debug 2023-12-07T17:24:11.531+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.008257+0000 front 2023-12-07T17:23:36.007792+0000 (oldest deadline 2023-12-07T17:24:01.907563+0000) 2023-12-07T17:24:11.819 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[35040]: debug 2023-12-07T17:24:11.531+0000 7fec8cb44700 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.009206+0000 front 2023-12-07T17:23:46.009169+0000 (oldest deadline 2023-12-07T17:24:08.908917+0000) 2023-12-07T17:24:11.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[57324]: debug 2023-12-07T17:24:11.535+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:11.819 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:11 smithi184 bash[57324]: debug 2023-12-07T17:24:11.535+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:12.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[67940]: debug 2023-12-07T17:24:12.235+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:12.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[67940]: debug 2023-12-07T17:24:12.235+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:12.765 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[45837]: debug 2023-12-07T17:24:12.651+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:12.765 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[45837]: debug 2023-12-07T17:24:12.651+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:12.765 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[57324]: debug 2023-12-07T17:24:12.563+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:12.766 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[57324]: debug 2023-12-07T17:24:12.563+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:12.945 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[84076]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-4 2023-12-07T17:24:13.227 INFO:journalctl@ceph.osd.4.smithi184.stdout:Dec 07 17:24:12 smithi184 bash[84144]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-4 2023-12-07T17:24:13.318 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[67940]: debug 2023-12-07T17:24:13.223+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:13.318 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[67940]: debug 2023-12-07T17:24:13.223+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:13.891 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[57324]: debug 2023-12-07T17:24:13.603+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:13.892 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[57324]: debug 2023-12-07T17:24:13.603+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:13.892 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[45837]: debug 2023-12-07T17:24:13.635+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:13.892 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:13 smithi184 bash[45837]: debug 2023-12-07T17:24:13.635+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:14.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[67940]: debug 2023-12-07T17:24:14.203+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:14.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[67940]: debug 2023-12-07T17:24:14.203+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:15.048 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[45837]: debug 2023-12-07T17:24:14.663+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:15.048 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[45837]: debug 2023-12-07T17:24:14.663+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:15.048 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[57324]: debug 2023-12-07T17:24:14.599+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:15.049 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:14 smithi184 bash[57324]: debug 2023-12-07T17:24:14.599+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:15.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.4.service' 2023-12-07T17:24:15.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:15.082 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-07T17:24:15.082 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-07T17:24:15.083 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.5 2023-12-07T17:24:15.306 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 systemd[1]: Stopping Ceph osd.5 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:15.307 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[67940]: debug 2023-12-07T17:24:15.167+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:15.307 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[67940]: debug 2023-12-07T17:24:15.167+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:15.568 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[45837]: debug 2023-12-07T17:24:15.299+0000 7f2b541fd700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:24:15.568 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[45837]: debug 2023-12-07T17:24:15.299+0000 7f2b541fd700 -1 osd.5 71 *** Got signal Terminated *** 2023-12-07T17:24:15.568 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[45837]: debug 2023-12-07T17:24:15.299+0000 7f2b541fd700 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:24:16.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[45837]: debug 2023-12-07T17:24:15.651+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:16.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[45837]: debug 2023-12-07T17:24:15.651+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:16.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[57324]: debug 2023-12-07T17:24:15.631+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:16.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:15 smithi184 bash[57324]: debug 2023-12-07T17:24:15.631+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:16.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[67940]: debug 2023-12-07T17:24:16.175+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:16.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[67940]: debug 2023-12-07T17:24:16.175+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:17.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[45837]: debug 2023-12-07T17:24:16.687+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:17.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[45837]: debug 2023-12-07T17:24:16.687+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:17.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[57324]: debug 2023-12-07T17:24:16.607+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:17.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:16 smithi184 bash[57324]: debug 2023-12-07T17:24:16.607+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:17.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[67940]: debug 2023-12-07T17:24:17.215+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:17.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[67940]: debug 2023-12-07T17:24:17.215+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:18.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[45837]: debug 2023-12-07T17:24:17.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:18.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[45837]: debug 2023-12-07T17:24:17.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:18.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[45837]: debug 2023-12-07T17:24:17.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:53.954017+0000 front 2023-12-07T17:23:53.954264+0000 (oldest deadline 2023-12-07T17:24:16.854066+0000) 2023-12-07T17:24:18.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[57324]: debug 2023-12-07T17:24:17.655+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:18.069 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:17 smithi184 bash[57324]: debug 2023-12-07T17:24:17.655+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:18.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[67940]: debug 2023-12-07T17:24:18.215+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:18.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[67940]: debug 2023-12-07T17:24:18.215+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:19.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[45837]: debug 2023-12-07T17:24:18.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:19.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[45837]: debug 2023-12-07T17:24:18.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:19.068 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[45837]: debug 2023-12-07T17:24:18.659+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:53.954017+0000 front 2023-12-07T17:23:53.954264+0000 (oldest deadline 2023-12-07T17:24:16.854066+0000) 2023-12-07T17:24:19.069 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[57324]: debug 2023-12-07T17:24:18.675+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:19.069 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:18 smithi184 bash[57324]: debug 2023-12-07T17:24:18.675+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:19.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[67940]: debug 2023-12-07T17:24:19.231+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:19.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[67940]: debug 2023-12-07T17:24:19.231+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:20.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[57324]: debug 2023-12-07T17:24:19.715+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:20.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[57324]: debug 2023-12-07T17:24:19.715+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:20.069 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[45837]: debug 2023-12-07T17:24:19.623+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.749947+0000 front 2023-12-07T17:23:37.750152+0000 (oldest deadline 2023-12-07T17:23:59.450072+0000) 2023-12-07T17:24:20.069 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[45837]: debug 2023-12-07T17:24:19.623+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:45.753081+0000 front 2023-12-07T17:23:45.753049+0000 (oldest deadline 2023-12-07T17:24:08.052864+0000) 2023-12-07T17:24:20.069 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:19 smithi184 bash[45837]: debug 2023-12-07T17:24:19.623+0000 7f2b4eff5700 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:53.954017+0000 front 2023-12-07T17:23:53.954264+0000 (oldest deadline 2023-12-07T17:24:16.854066+0000) 2023-12-07T17:24:20.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:20 smithi184 bash[67940]: debug 2023-12-07T17:24:20.263+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:20.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:20 smithi184 bash[67940]: debug 2023-12-07T17:24:20.263+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:21.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:20 smithi184 bash[57324]: debug 2023-12-07T17:24:20.663+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:21.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:20 smithi184 bash[57324]: debug 2023-12-07T17:24:20.663+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:21.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:20 smithi184 bash[57324]: debug 2023-12-07T17:24:20.663+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:21.516 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[84285]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-5 2023-12-07T17:24:21.516 INFO:journalctl@ceph.osd.5.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[84353]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-5 2023-12-07T17:24:21.517 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[67940]: debug 2023-12-07T17:24:21.239+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:21.517 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[67940]: debug 2023-12-07T17:24:21.239+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:21.517 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[67940]: debug 2023-12-07T17:24:21.239+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:21.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[57324]: debug 2023-12-07T17:24:21.639+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:21.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[57324]: debug 2023-12-07T17:24:21.639+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:21.818 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:21 smithi184 bash[57324]: debug 2023-12-07T17:24:21.639+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:22.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[67940]: debug 2023-12-07T17:24:22.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:22.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[67940]: debug 2023-12-07T17:24:22.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:22.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[67940]: debug 2023-12-07T17:24:22.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:23.023 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[57324]: debug 2023-12-07T17:24:22.651+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:23.023 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[57324]: debug 2023-12-07T17:24:22.651+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:23.023 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:22 smithi184 bash[57324]: debug 2023-12-07T17:24:22.651+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:23.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.5.service' 2023-12-07T17:24:23.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:23.469 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-07T17:24:23.470 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-07T17:24:23.470 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.6 2023-12-07T17:24:23.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 systemd[1]: Stopping Ceph osd.6 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:23.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[67940]: debug 2023-12-07T17:24:23.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:23.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[67940]: debug 2023-12-07T17:24:23.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:23.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[67940]: debug 2023-12-07T17:24:23.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:24.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.607+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:24.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.607+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:24.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.607+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:24.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.679+0000 7ff79fd80700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:24:24.069 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.679+0000 7ff79fd80700 -1 osd.6 71 *** Got signal Terminated *** 2023-12-07T17:24:24.069 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:23 smithi184 bash[57324]: debug 2023-12-07T17:24:23.679+0000 7ff79fd80700 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:24:24.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[67940]: debug 2023-12-07T17:24:24.299+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:24.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[67940]: debug 2023-12-07T17:24:24.299+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:24.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[67940]: debug 2023-12-07T17:24:24.299+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:25.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[57324]: debug 2023-12-07T17:24:24.563+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:25.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[57324]: debug 2023-12-07T17:24:24.563+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:25.068 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:24 smithi184 bash[57324]: debug 2023-12-07T17:24:24.563+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:25.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[57324]: debug 2023-12-07T17:24:25.519+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:25.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[57324]: debug 2023-12-07T17:24:25.519+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:25.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[57324]: debug 2023-12-07T17:24:25.519+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:25.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[67940]: debug 2023-12-07T17:24:25.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:25.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[67940]: debug 2023-12-07T17:24:25.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:25.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:25 smithi184 bash[67940]: debug 2023-12-07T17:24:25.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:26.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[57324]: debug 2023-12-07T17:24:26.495+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:26.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[57324]: debug 2023-12-07T17:24:26.495+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[57324]: debug 2023-12-07T17:24:26.495+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[57324]: debug 2023-12-07T17:24:26.495+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.885782+0000 front 2023-12-07T17:24:00.885738+0000 (oldest deadline 2023-12-07T17:24:26.185524+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[67940]: debug 2023-12-07T17:24:26.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[67940]: debug 2023-12-07T17:24:26.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[67940]: debug 2023-12-07T17:24:26.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:26.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:26 smithi184 bash[67940]: debug 2023-12-07T17:24:26.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:27.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[57324]: debug 2023-12-07T17:24:27.451+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:27.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[57324]: debug 2023-12-07T17:24:27.451+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:27.569 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[57324]: debug 2023-12-07T17:24:27.451+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:27.569 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[57324]: debug 2023-12-07T17:24:27.451+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.885782+0000 front 2023-12-07T17:24:00.885738+0000 (oldest deadline 2023-12-07T17:24:26.185524+0000) 2023-12-07T17:24:27.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[67940]: debug 2023-12-07T17:24:27.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:27.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[67940]: debug 2023-12-07T17:24:27.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:27.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[67940]: debug 2023-12-07T17:24:27.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:27.570 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:27 smithi184 bash[67940]: debug 2023-12-07T17:24:27.279+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:28.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[57324]: debug 2023-12-07T17:24:28.419+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:37.179607+0000 front 2023-12-07T17:23:37.179933+0000 (oldest deadline 2023-12-07T17:24:02.479475+0000) 2023-12-07T17:24:28.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[57324]: debug 2023-12-07T17:24:28.419+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:43.581265+0000 front 2023-12-07T17:23:43.581360+0000 (oldest deadline 2023-12-07T17:24:07.080979+0000) 2023-12-07T17:24:28.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[57324]: debug 2023-12-07T17:24:28.419+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:54.484368+0000 front 2023-12-07T17:23:54.484296+0000 (oldest deadline 2023-12-07T17:24:19.784183+0000) 2023-12-07T17:24:28.568 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[57324]: debug 2023-12-07T17:24:28.419+0000 7ff798373700 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.885782+0000 front 2023-12-07T17:24:00.885738+0000 (oldest deadline 2023-12-07T17:24:26.185524+0000) 2023-12-07T17:24:28.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[67940]: debug 2023-12-07T17:24:28.271+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:28.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[67940]: debug 2023-12-07T17:24:28.271+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:28.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[67940]: debug 2023-12-07T17:24:28.271+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:28.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:28 smithi184 bash[67940]: debug 2023-12-07T17:24:28.271+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:29.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[67940]: debug 2023-12-07T17:24:29.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:29.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[67940]: debug 2023-12-07T17:24:29.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:29.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[67940]: debug 2023-12-07T17:24:29.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:29.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[67940]: debug 2023-12-07T17:24:29.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:29.881 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[84493]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-6 2023-12-07T17:24:29.881 INFO:journalctl@ceph.osd.6.smithi184.stdout:Dec 07 17:24:29 smithi184 bash[84561]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-6 2023-12-07T17:24:30.631 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:30 smithi184 bash[67940]: debug 2023-12-07T17:24:30.331+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:30.632 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:30 smithi184 bash[67940]: debug 2023-12-07T17:24:30.331+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:30.632 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:30 smithi184 bash[67940]: debug 2023-12-07T17:24:30.331+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:30.632 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:30 smithi184 bash[67940]: debug 2023-12-07T17:24:30.331+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:31.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:31 smithi184 bash[67940]: debug 2023-12-07T17:24:31.371+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:31.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:31 smithi184 bash[67940]: debug 2023-12-07T17:24:31.371+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:31.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:31 smithi184 bash[67940]: debug 2023-12-07T17:24:31.371+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:31.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:31 smithi184 bash[67940]: debug 2023-12-07T17:24:31.371+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:31.783 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.6.service' 2023-12-07T17:24:31.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:31.816 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-07T17:24:31.816 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-07T17:24:31.816 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.7 2023-12-07T17:24:32.035 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:31 smithi184 systemd[1]: Stopping Ceph osd.7 for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:32.318 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.027+0000 7f722993d700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2023-12-07T17:24:32.318 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.027+0000 7f722993d700 -1 osd.7 71 *** Got signal Terminated *** 2023-12-07T17:24:32.318 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.027+0000 7f722993d700 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T17:24:32.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.327+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:32.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.327+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:32.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.327+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:32.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:32 smithi184 bash[67940]: debug 2023-12-07T17:24:32.327+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:33.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[67940]: debug 2023-12-07T17:24:33.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:33.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[67940]: debug 2023-12-07T17:24:33.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:33.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[67940]: debug 2023-12-07T17:24:33.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:33.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[67940]: debug 2023-12-07T17:24:33.295+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[83299]: ts=2023-12-07T17:24:33.890Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:24:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[83299]: ts=2023-12-07T17:24:33.893Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:24:34.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[83299]: ts=2023-12-07T17:24:33.894Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:24:34.319 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:33 smithi184 bash[83299]: ts=2023-12-07T17:24:33.896Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.64:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.64:8765: connect: connection refused" 2023-12-07T17:24:34.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:34 smithi184 bash[67940]: debug 2023-12-07T17:24:34.311+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:34.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:34 smithi184 bash[67940]: debug 2023-12-07T17:24:34.311+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:34.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:34 smithi184 bash[67940]: debug 2023-12-07T17:24:34.311+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:34.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:34 smithi184 bash[67940]: debug 2023-12-07T17:24:34.311+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:34.818 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:34 smithi184 bash[67940]: debug 2023-12-07T17:24:34.311+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.184:6804 osd.4 since back 2023-12-07T17:24:11.495556+0000 front 2023-12-07T17:24:11.495694+0000 (oldest deadline 2023-12-07T17:24:33.795224+0000) 2023-12-07T17:24:35.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:35 smithi184 bash[67940]: debug 2023-12-07T17:24:35.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:35.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:35 smithi184 bash[67940]: debug 2023-12-07T17:24:35.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:35.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:35 smithi184 bash[67940]: debug 2023-12-07T17:24:35.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:35.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:35 smithi184 bash[67940]: debug 2023-12-07T17:24:35.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:35.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:35 smithi184 bash[67940]: debug 2023-12-07T17:24:35.287+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.184:6804 osd.4 since back 2023-12-07T17:24:11.495556+0000 front 2023-12-07T17:24:11.495694+0000 (oldest deadline 2023-12-07T17:24:33.795224+0000) 2023-12-07T17:24:36.568 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:36 smithi184 bash[67940]: debug 2023-12-07T17:24:36.247+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-12-07T17:23:36.290924+0000 front 2023-12-07T17:23:36.291133+0000 (oldest deadline 2023-12-07T17:24:02.190509+0000) 2023-12-07T17:24:36.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:36 smithi184 bash[67940]: debug 2023-12-07T17:24:36.247+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6814 osd.1 since back 2023-12-07T17:23:46.292189+0000 front 2023-12-07T17:23:46.292264+0000 (oldest deadline 2023-12-07T17:24:08.591584+0000) 2023-12-07T17:24:36.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:36 smithi184 bash[67940]: debug 2023-12-07T17:24:36.247+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6822 osd.2 since back 2023-12-07T17:23:50.293337+0000 front 2023-12-07T17:23:50.293210+0000 (oldest deadline 2023-12-07T17:24:20.893456+0000) 2023-12-07T17:24:36.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:36 smithi184 bash[67940]: debug 2023-12-07T17:24:36.247+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.64:6830 osd.3 since back 2023-12-07T17:24:00.894337+0000 front 2023-12-07T17:24:00.894314+0000 (oldest deadline 2023-12-07T17:24:25.594089+0000) 2023-12-07T17:24:36.569 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:36 smithi184 bash[67940]: debug 2023-12-07T17:24:36.247+0000 7f7223733700 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.184:6804 osd.4 since back 2023-12-07T17:24:11.495556+0000 front 2023-12-07T17:24:11.495694+0000 (oldest deadline 2023-12-07T17:24:33.795224+0000) 2023-12-07T17:24:38.199 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:37 smithi184 bash[84702]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-7 2023-12-07T17:24:38.200 INFO:journalctl@ceph.osd.7.smithi184.stdout:Dec 07 17:24:38 smithi184 bash[84770]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-osd-7 2023-12-07T17:24:40.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@osd.7.service' 2023-12-07T17:24:40.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:40.077 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-07T17:24:40.078 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2023-12-07T17:24:40.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a 2023-12-07T17:24:40.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@rgw.foo.a.service' 2023-12-07T17:24:40.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:40.169 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2023-12-07T17:24:40.169 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2023-12-07T17:24:40.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-7ae18f76-9522-11ee-95a2-87774f69a715@prometheus.a 2023-12-07T17:24:40.318 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 systemd[1]: Stopping Ceph prometheus.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:40.818 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.321Z caller=main.go:907 level=info msg="Scrape manager stopped" 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.322Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.322Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2023-12-07T17:24:40.819 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Dec 07 17:24:40 smithi184 bash[83299]: ts=2023-12-07T17:24:40.322Z caller=main.go:1147 level=info msg="See you next time!" 2023-12-07T17:24:41.046 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ae18f76-9522-11ee-95a2-87774f69a715@prometheus.a.service' 2023-12-07T17:24:41.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T17:24:41.073 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2023-12-07T17:24:41.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-07T17:24:41.245 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:24:48.121 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:47 smithi064 systemd[1]: Stopping Ceph node-exporter.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:48.871 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:48 smithi064 bash[107010]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-node-exporter-a 2023-12-07T17:24:48.871 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:48 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2023-12-07T17:24:48.871 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:48 smithi064 bash[107074]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-node-exporter-a 2023-12-07T17:24:48.871 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:48 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2023-12-07T17:24:48.871 INFO:journalctl@ceph.node-exporter.a.smithi064.stdout:Dec 07 17:24:48 smithi064 systemd[1]: Stopped Ceph node-exporter.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:24:49.185 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:48 smithi064 systemd[1]: Stopping Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:49.186 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:49 smithi064 bash[96727]: ts=2023-12-07T17:24:49.076Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2023-12-07T17:24:50.089 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:49 smithi064 bash[107136]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-alertmanager-a 2023-12-07T17:24:50.089 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:49 smithi064 bash[107201]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-alertmanager-a 2023-12-07T17:24:50.089 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:49 smithi064 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@alertmanager.a.service: Succeeded. 2023-12-07T17:24:50.089 INFO:journalctl@ceph.alertmanager.a.smithi064.stdout:Dec 07 17:24:49 smithi064 systemd[1]: Stopped Ceph alertmanager.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:24:50.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-07T17:24:50.792 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:24:57.046 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:24:56 smithi184 systemd[1]: Stopping Ceph iscsi.iscsi.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:24:57.318 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:24:57 smithi184 bash[79431]: debug Shutdown received 2023-12-07T17:25:07.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:07 smithi184 bash[85210]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-iscsi-iscsi-a 2023-12-07T17:25:08.064 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:07 smithi184 bash[85293]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-iscsi-iscsi-a 2023-12-07T17:25:08.064 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:07 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2023-12-07T17:25:08.641 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:08 smithi184 bash[85302]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-iscsi-iscsi-a-tcmu 2023-12-07T17:25:08.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:08 smithi184 bash[85338]: rm: cannot remove '/run/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@iscsi.iscsi.a.tcmu.service-pid': No such file or directory 2023-12-07T17:25:08.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:08 smithi184 bash[85339]: rm: cannot remove '/run/ceph-7ae18f76-9522-11ee-95a2-87774f69a715@iscsi.iscsi.a.tcmu.service-cid': No such file or directory 2023-12-07T17:25:08.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:08 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2023-12-07T17:25:08.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Dec 07 17:25:08 smithi184 systemd[1]: Stopped Ceph iscsi.iscsi.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:25:09.068 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:08 smithi184 systemd[1]: Stopping Ceph node-exporter.b for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:25:10.042 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:09 smithi184 bash[85422]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-node-exporter-b 2023-12-07T17:25:10.042 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:09 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2023-12-07T17:25:10.042 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:09 smithi184 bash[85486]: Error response from daemon: No such container: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-node-exporter-b 2023-12-07T17:25:10.042 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:09 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2023-12-07T17:25:10.042 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Dec 07 17:25:09 smithi184 systemd[1]: Stopped Ceph node-exporter.b for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:25:10.318 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 systemd[1]: Stopping Ceph grafana.a for 7ae18f76-9522-11ee-95a2-87774f69a715... 2023-12-07T17:25:11.036 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 bash[85549]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-grafana-a 2023-12-07T17:25:11.036 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 bash[85613]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715-grafana-a 2023-12-07T17:25:11.036 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2023-12-07T17:25:11.037 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 systemd[1]: ceph-7ae18f76-9522-11ee-95a2-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2023-12-07T17:25:11.037 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Dec 07 17:25:10 smithi184 systemd[1]: Stopped Ceph grafana.a for 7ae18f76-9522-11ee-95a2-87774f69a715. 2023-12-07T17:25:11.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:25:11.578 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T17:25:11.593 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-07T17:25:11.594 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi064/crash 2023-12-07T17:25:11.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash -- . 2023-12-07T17:25:11.632 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-07T17:25:11.632 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2023-12-07T17:25:11.634 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi184/crash 2023-12-07T17:25:11.635 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash -- . 2023-12-07T17:25:11.650 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-07T17:25:11.651 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2023-12-07T17:25:11.652 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-07T17:25:11.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-12-07T17:25:11.691 INFO:teuthology.orchestra.run.smithi064.stderr:grep: /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-12-07T17:25:11.692 INFO:tasks.cephadm:Compressing logs... 2023-12-07T17:25:11.692 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-07T17:25:11.736 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-07T17:25:11.749 INFO:teuthology.orchestra.run.smithi064.stderr:find: '/var/log/rbd-target-api': No such file or directory 2023-12-07T17:25:11.753 INFO:teuthology.orchestra.run.smithi184.stderr:find: '/var/log/rbd-target-api': No such file or directory 2023-12-07T17:25:11.759 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-07T17:25:11.760 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.a.log 2023-12-07T17:25:11.760 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-client.rgw.foo.a.log 2023-12-07T17:25:11.760 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.y.log 2023-12-07T17:25:11.761 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-07T17:25:11.761 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.c.log 2023-12-07T17:25:11.761 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.1.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.b.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.5.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.x.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.b.log: /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.6.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/tcmu-runner.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.4.log 2023-12-07T17:25:11.764 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.0.log 2023-12-07T17:25:11.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.3.log 2023-12-07T17:25:11.765 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-07T17:25:11.766 INFO:teuthology.orchestra.run.smithi184.stderr:gzip /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/tcmu-runner.log: -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-07T17:25:11.766 INFO:teuthology.orchestra.run.smithi184.stderr: 67.5% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/tcmu-runner.log.gz 2023-12-07T17:25:11.766 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.7.log 2023-12-07T17:25:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.3.log: 91.1% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-07T17:25:11.767 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.2.log 2023-12-07T17:25:11.771 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log: /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.7.log: 84.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-07T17:25:11.774 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log: 90.4% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-client.rgw.foo.a.log.gz 2023-12-07T17:25:11.776 INFO:teuthology.orchestra.run.smithi184.stderr: 91.8% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.x.log.gz 2023-12-07T17:25:11.786 INFO:teuthology.orchestra.run.smithi184.stderr: 90.5% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-07T17:25:11.788 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.2.log: 90.6% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-07T17:25:11.916 INFO:teuthology.orchestra.run.smithi184.stderr: 92.0% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.4.log.gz 2023-12-07T17:25:11.931 INFO:teuthology.orchestra.run.smithi184.stderr: 92.5% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.5.log.gz 2023-12-07T17:25:11.942 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.7.log.gz 2023-12-07T17:25:11.980 INFO:teuthology.orchestra.run.smithi064.stderr: 89.7% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mgr.y.log.gz 2023-12-07T17:25:12.001 INFO:teuthology.orchestra.run.smithi184.stderr: 92.5% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.6.log.gz 2023-12-07T17:25:12.003 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.c.log.gz 2023-12-07T17:25:12.014 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.3.log.gz 2023-12-07T17:25:12.028 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.2.log.gz 2023-12-07T17:25:12.040 INFO:teuthology.orchestra.run.smithi184.stderr: 92.4% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.b.log.gz 2023-12-07T17:25:12.041 INFO:teuthology.orchestra.run.smithi184.stderr: 2023-12-07T17:25:12.041 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m0.300s 2023-12-07T17:25:12.041 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m1.029s 2023-12-07T17:25:12.041 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.056s 2023-12-07T17:25:12.055 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.0.log.gz 2023-12-07T17:25:12.102 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-osd.1.log.gz 2023-12-07T17:25:12.392 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/7ae18f76-9522-11ee-95a2-87774f69a715/ceph-mon.a.log.gz 2023-12-07T17:25:12.394 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-12-07T17:25:12.394 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m0.654s 2023-12-07T17:25:12.395 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m2.207s 2023-12-07T17:25:12.395 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.088s 2023-12-07T17:25:12.395 INFO:tasks.cephadm:Archiving logs... 2023-12-07T17:25:12.395 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi064/log 2023-12-07T17:25:12.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-07T17:25:12.740 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi184/log 2023-12-07T17:25:12.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-07T17:25:12.910 INFO:tasks.cephadm:Removing cluster... 2023-12-07T17:25:12.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 --force 2023-12-07T17:25:13.081 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:25:14.291 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ae18f76-9522-11ee-95a2-87774f69a715 --force 2023-12-07T17:25:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 7ae18f76-9522-11ee-95a2-87774f69a715 2023-12-07T17:25:15.657 INFO:tasks.cephadm:Removing cephadm ... 2023-12-07T17:25:15.657 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-07T17:25:15.665 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-07T17:25:15.672 INFO:tasks.cephadm:Teardown complete 2023-12-07T17:25:15.672 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/cephadm.py", line 1791, in task healthy(ctx=ctx, config=config) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph.py", line 1478, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph_manager.py", line 3185, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-12-07T17:25:15.747 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9fc5fce1f3ff4ca39ad364134425fce2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/cephadm.py", line 1791, in task healthy(ctx=ctx, config=config) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph.py", line 1478, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_e068ebcf7ecc6503f24666fb6b152034d3fe1067/qa/tasks/ceph_manager.py", line 3185, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-12-07T17:25:15.750 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-07T17:25:15.759 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2023-12-07T17:25:15.776 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi064:/dev/vg_nvme/lv_1... 2023-12-07T17:25:15.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo nvme disconnect -n lv_1 2023-12-07T17:25:16.095 INFO:teuthology.orchestra.run.smithi064.stdout:NQN:lv_1 disconnected 1 controller(s) 2023-12-07T17:25:16.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:16.097 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi064:/dev/vg_nvme/lv_2... 2023-12-07T17:25:16.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo nvme disconnect -n lv_2 2023-12-07T17:25:16.407 INFO:teuthology.orchestra.run.smithi064.stdout:NQN:lv_2 disconnected 1 controller(s) 2023-12-07T17:25:16.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:16.409 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi064:/dev/vg_nvme/lv_3... 2023-12-07T17:25:16.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo nvme disconnect -n lv_3 2023-12-07T17:25:16.691 INFO:teuthology.orchestra.run.smithi064.stdout:NQN:lv_3 disconnected 1 controller(s) 2023-12-07T17:25:16.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:16.693 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi064:/dev/vg_nvme/lv_4... 2023-12-07T17:25:16.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo nvme disconnect -n lv_4 2023-12-07T17:25:16.963 INFO:teuthology.orchestra.run.smithi064.stdout:NQN:lv_4 disconnected 1 controller(s) 2023-12-07T17:25:16.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:16.965 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-12-07T17:25:16.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/scratch_devs 2023-12-07T17:25:16.980 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi184:/dev/vg_nvme/lv_1... 2023-12-07T17:25:16.980 DEBUG:teuthology.orchestra.run.smithi184:> sudo nvme disconnect -n lv_1 2023-12-07T17:25:17.261 INFO:teuthology.orchestra.run.smithi184.stdout:NQN:lv_1 disconnected 1 controller(s) 2023-12-07T17:25:17.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:17.263 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi184:/dev/vg_nvme/lv_2... 2023-12-07T17:25:17.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo nvme disconnect -n lv_2 2023-12-07T17:25:17.601 INFO:teuthology.orchestra.run.smithi184.stdout:NQN:lv_2 disconnected 1 controller(s) 2023-12-07T17:25:17.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:17.603 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi184:/dev/vg_nvme/lv_3... 2023-12-07T17:25:17.603 DEBUG:teuthology.orchestra.run.smithi184:> sudo nvme disconnect -n lv_3 2023-12-07T17:25:17.917 INFO:teuthology.orchestra.run.smithi184.stdout:NQN:lv_3 disconnected 1 controller(s) 2023-12-07T17:25:17.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:17.919 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi184:/dev/vg_nvme/lv_4... 2023-12-07T17:25:17.919 DEBUG:teuthology.orchestra.run.smithi184:> sudo nvme disconnect -n lv_4 2023-12-07T17:25:18.217 INFO:teuthology.orchestra.run.smithi184.stdout:NQN:lv_4 disconnected 1 controller(s) 2023-12-07T17:25:18.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:18.219 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-12-07T17:25:18.219 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/scratch_devs 2023-12-07T17:25:18.234 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-07T17:25:18.245 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-07T17:25:18.245 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T17:25:18.248 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T17:25:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-12-07T17:25:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-12-07T17:25:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:*hv01.front.sepi 44.4.53.2 2 u 29 64 377 0.098 0.799 0.151 2023-12-07T17:25:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:+hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.088 1.707 0.153 2023-12-07T17:25:18.267 INFO:teuthology.orchestra.run.smithi064.stdout:+hv03.front.sepi 74.6.168.72 3 u 20 64 377 0.088 1.738 0.256 2023-12-07T17:25:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-12-07T17:25:18.296 INFO:teuthology.orchestra.run.smithi184.stdout: remote refid st t when poll reach delay offset jitter 2023-12-07T17:25:18.296 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================== 2023-12-07T17:25:18.296 INFO:teuthology.orchestra.run.smithi184.stdout:*hv01.front.sepi 44.4.53.2 2 u 35 64 377 0.058 0.470 0.297 2023-12-07T17:25:18.296 INFO:teuthology.orchestra.run.smithi184.stdout:+hv02.front.sepi 63.231.80.2 3 u 15 64 377 0.090 1.648 0.642 2023-12-07T17:25:18.297 INFO:teuthology.orchestra.run.smithi184.stdout:+hv03.front.sepi 74.6.168.72 3 u 29 64 377 0.088 0.981 0.209 2023-12-07T17:25:18.297 INFO:teuthology.orchestra.run.smithi184.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-12-07T17:25:18.298 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-07T17:25:18.306 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-07T17:25:18.307 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-07T17:25:18.315 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-07T17:25:18.365 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-07T17:25:18.374 INFO:teuthology.task.internal:Duration was 2040.320008 seconds 2023-12-07T17:25:18.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-07T17:25:18.383 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-07T17:25:18.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-07T17:25:18.386 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-07T17:25:18.438 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-07T17:25:18.438 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-12-07T17:25:18.438 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-07T17:25:18.452 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2023-12-07T17:25:18.453 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-07T17:25:18.468 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-07T17:25:18.468 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-07T17:25:18.496 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-07T17:25:18.555 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-07T17:25:18.556 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-07T17:25:18.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-07T17:25:18.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-07T17:25:18.689 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-07T17:25:18.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-07T17:25:18.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-07T17:25:18.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-07T17:25:18.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-07T17:25:18.749 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-07T17:25:18.761 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-12-07T17:25:18.775 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2023-12-07T17:25:18.799 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-07T17:25:18.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:18.832 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-07T17:25:18.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T17:25:18.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-07T17:25:18.856 INFO:teuthology.task.internal:Transferring archived files... 2023-12-07T17:25:18.857 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi064 2023-12-07T17:25:18.857 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-07T17:25:18.905 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193/remote/smithi184 2023-12-07T17:25:18.905 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-07T17:25:18.930 INFO:teuthology.task.internal:Removing archive directory... 2023-12-07T17:25:18.931 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-07T17:25:18.944 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-07T17:25:18.975 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-07T17:25:18.985 INFO:teuthology.task.internal:Not uploading archives. 2023-12-07T17:25:18.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-07T17:25:18.998 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-07T17:25:18.999 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-07T17:25:19.001 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-07T17:25:19.007 INFO:teuthology.orchestra.run.smithi064.stdout: 1048618 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 7 17:25 /home/ubuntu/cephtest 2023-12-07T17:25:19.024 INFO:teuthology.orchestra.run.smithi184.stdout: 1048603 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 7 17:25 /home/ubuntu/cephtest 2023-12-07T17:25:19.026 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-07T17:25:19.038 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-07T17:25:19.085 INFO:teuthology.nuke:Checking targets against current locks 2023-12-07T17:25:19.114 DEBUG:teuthology.nuke:shortname: smithi064 2023-12-07T17:25:19.115 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T17:25:19.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 16:39:31.238400', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-12-07T17:25:19.145 DEBUG:teuthology.nuke:shortname: smithi184 2023-12-07T17:25:19.145 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T17:25:19.171 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-07_16:37:24-rados-wip-yuri8-testing-2023-12-06-1425-distro-default-smithi/7482193', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 16:39:31.237159', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-12-07T17:25:19.194 INFO:teuthology.orchestra.console.smithi064:Power off 2023-12-07T17:25:19.194 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-07T17:25:19.223 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-12-07T17:25:19.223 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:19.243 INFO:teuthology.orchestra.console.smithi184:Power off 2023-12-07T17:25:19.243 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-07T17:25:19.255 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-12-07T17:25:19.269 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2023-12-07T17:25:19.270 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:19.297 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-12-07T17:25:23.257 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:23.299 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:23.367 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-12-07T17:25:23.411 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-12-07T17:25:27.370 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:27.412 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:27.482 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-12-07T17:25:27.526 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-12-07T17:25:27.583 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-12-07T17:25:31.530 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T17:25:31.641 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2023-12-07T17:25:31.742 INFO:teuthology.orchestra.console.smithi184:Power off completed 2023-12-07T17:25:31.846 INFO:teuthology.run:Summary data: description: rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 2040.3200075626373 failure_reason: timeout expired in wait_until_healthy owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9fc5fce1f3ff4ca39ad364134425fce2 status: fail success: false 2023-12-07T17:25:31.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-07T17:25:31.920 INFO:teuthology.run:FAIL