2021-02-20T07:45:35.254 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-20T07:45:35.256 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_trociny@teuthology --archive /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891 --name trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} --verbose -- /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891/orig.config.yaml 2021-02-20T07:45:35.311 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-02-20T07:45:35.371 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891 branch: wip-mgolub-testing description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} email: to.my.trociny@gmail.com first_in_suite: false job_id: '5897891' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-mgolub-testing ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: simple client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs log-ignorelist: - \(OSD_SLOW_PING_TIME log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mgolub-testing sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b owner: scheduled_trociny@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b sleep_before_teardown: 0 suite: rbd suite_branch: wip-mgolub-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b targets: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1mnk0O6+NreOssSFrZ95wpiF8tx9IRDQM3nDni9DDXQbHBttgEFNAoVR5+7i1QGU1IjrmGogPN1ot03XpA9AbaT/KvymYBq2iL5O5eTwSJ6OAPai1vNH2KD91ISBja7+c9LmcoTu69zflvewYGWRD2Av8vnC3JEGqC/c1ROTnFaxOAmIz8OTxmN7XxxRmq/D2QOBvFFdV1MsshGIPmFGWj7MgYSDZoe7TWpjXJo3L1HG9fa79HoOFSrsn/FvBTQ772Y0YluDkmbNsGXUuHB09bchr1XBIQNqbnlZv/Mx2PschkZIBUc2Ob5d3jpWNjvjJVt7pzVutl4PGvip1eWlL smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDstIYH2/PVs6ZJhqo/CwHh6FWOoxqMGz3sP3ZIantQz95ZbsZA0mJ/1DV7UlYIghJhw3XsASkRnCEPgYH0yPIU7PeG4IyWjvpEbKNNUXjUmPsEENYZL4IIIY6xFkuKZZpWRvS1WKebQmAPuGWyQAXS2E+Xh0WGQIcy2kC2pPEICjmpjjH8dvtN9+i3xDcbcUZ7T/EnJh+KeCtTPQEFs2kdZCP4m4DMG0hiwqlwPWWgnA2QXMLguc4v19RWDzot2lhDXNyRcq9PGghJifN+R5UbgKRssiEd4UxM44CL1MSdo89fSy04L0VpoHgnzeUn9xhYFodHp1p0EPC9laBn5rv/ tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_IMAGE_MODE: snapshot MIRROR_POOL_MODE: image RBD_IMAGE_FEATURES: layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.29353 2021-02-20T07:45:35.371 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa; will attempt to use it 2021-02-20T07:45:35.372 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa/tasks 2021-02-20T07:45:35.372 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-02-20T07:45:35.375 INFO:teuthology.task.internal:Checking packages... 2021-02-20T07:45:35.403 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'd318d42d502ea13c78f2af2499efd19dcc7e174b' 2021-02-20T07:45:35.403 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-02-20T07:45:35.404 INFO:teuthology.packaging:ref: None 2021-02-20T07:45:35.405 INFO:teuthology.packaging:tag: None 2021-02-20T07:45:35.405 INFO:teuthology.packaging:branch: wip-mgolub-testing 2021-02-20T07:45:35.406 INFO:teuthology.packaging:sha1: d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:45:35.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-mgolub-testing 2021-02-20T07:45:35.596 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:45:35.596 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-02-20T07:45:35.606 INFO:teuthology.task.internal:no buildpackages task found 2021-02-20T07:45:35.607 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-02-20T07:45:35.620 INFO:teuthology.task.internal:Saving configuration 2021-02-20T07:45:35.639 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-02-20T07:45:35.649 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-20T07:45:35.668 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 07:40:23.336505', 'description': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN60+RkuFh+HnUyw6VTTAAeJU/A9Y8HB8NX6PisbePI9', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi106.front.sepia.ceph.com'} 2021-02-20T07:45:35.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-20 07:40:23.336483', 'description': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891', 'locked_by': 'scheduled_trociny@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGYLcG1sIyclLjO4H2tmepOYnvVHx/S8ACXuuOrISLWZ', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi114.front.sepia.ceph.com'} 2021-02-20T07:45:35.687 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-02-20T07:45:35.698 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-02-20T07:45:35.699 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-02-20T07:45:35.699 INFO:teuthology.run_tasks:Running task console_log... 2021-02-20T07:45:35.764 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f866f432268>, signals=[15]) 2021-02-20T07:45:35.765 INFO:teuthology.run_tasks:Running task internal.connect... 2021-02-20T07:45:35.775 INFO:teuthology.task.internal:Opening connections... 2021-02-20T07:45:35.775 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:45:35.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:45:35.898 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:45:35.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:45:35.976 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-02-20T07:45:35.987 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2021-02-20T07:45:36.097 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2021-02-20T07:45:36.098 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2021-02-20T07:45:36.146 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Ubuntu" 2021-02-20T07:45:36.146 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T07:45:36.147 INFO:teuthology.orchestra.run.smithi106.stdout:ID=ubuntu 2021-02-20T07:45:36.147 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE=debian 2021-02-20T07:45:36.147 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T07:45:36.147 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="18.04" 2021-02-20T07:45:36.148 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T07:45:36.148 INFO:teuthology.orchestra.run.smithi106.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T07:45:36.148 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T07:45:36.149 INFO:teuthology.orchestra.run.smithi106.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T07:45:36.149 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_CODENAME=bionic 2021-02-20T07:45:36.149 INFO:teuthology.orchestra.run.smithi106.stdout:UBUNTU_CODENAME=bionic 2021-02-20T07:45:36.150 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2021-02-20T07:45:36.165 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2021-02-20T07:45:36.177 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2021-02-20T07:45:36.177 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2021-02-20T07:45:36.224 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2021-02-20T07:45:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-20T07:45:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2021-02-20T07:45:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2021-02-20T07:45:36.226 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-20T07:45:36.226 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="18.04" 2021-02-20T07:45:36.226 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-20T07:45:36.226 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-20T07:45:36.227 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-20T07:45:36.227 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-20T07:45:36.227 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=bionic 2021-02-20T07:45:36.227 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=bionic 2021-02-20T07:45:36.228 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2021-02-20T07:45:36.243 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-02-20T07:45:36.258 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-02-20T07:45:36.271 INFO:teuthology.task.internal:Checking for old test directory... 2021-02-20T07:45:36.271 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2021-02-20T07:45:36.281 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2021-02-20T07:45:36.286 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-02-20T07:45:36.295 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-02-20T07:45:36.296 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2021-02-20T07:45:36.326 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2021-02-20T07:45:36.411 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-02-20T07:45:36.456 INFO:teuthology.run_tasks:Running task kernel... 2021-02-20T07:45:36.474 DEBUG:teuthology.task.kernel:normalized config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.6': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.6': {'kdb': True, 'sha1': 'distro'}} 2021-02-20T07:45:36.475 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-02-20T07:45:36.475 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2021-02-20T07:45:36.482 INFO:teuthology.orchestra.run.smithi106.stdout:4.15.0-134-generic 2021-02-20T07:45:36.483 INFO:teuthology.task.kernel:Running kernel on smithi106: 4.15.0-134-generic 2021-02-20T07:45:36.484 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get clean 2021-02-20T07:45:36.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2021-02-20T07:45:36.845 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:45:36.846 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T07:45:36.912 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T07:45:37.168 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T07:45:37.245 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T07:45:37.383 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T07:45:37.632 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T07:45:37.737 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T07:45:37.752 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T07:45:37.762 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T07:45:37.763 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T07:45:37.785 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T07:45:37.806 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T07:45:37.818 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T07:45:37.829 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T07:45:37.836 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T07:45:37.836 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T07:45:37.837 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T07:45:37.908 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T07:45:37.950 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T07:45:37.955 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T07:45:37.955 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T07:45:37.956 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T07:45:39.437 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 13.1 MB in 3s (5,108 kB/s) 2021-02-20T07:45:40.834 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:45:40.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:45:40.962 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:45:41.203 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2021-02-20T07:45:41.204 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2021-02-20T07:45:41.287 INFO:teuthology.orchestra.run.smithi106.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:45:41.289 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:45:41.290 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:45:41.370 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:45:41.621 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2021-02-20T07:45:41.621 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2021-02-20T07:45:41.793 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2021-02-20T07:45:41.794 INFO:teuthology.orchestra.run.smithi106.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:45:41.794 INFO:teuthology.orchestra.run.smithi106.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T07:45:41.795 INFO:teuthology.orchestra.run.smithi106.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T07:45:41.796 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2021-02-20T07:45:41.796 INFO:teuthology.orchestra.run.smithi106.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T07:45:41.796 INFO:teuthology.orchestra.run.smithi106.stdout:Recommended packages: 2021-02-20T07:45:41.796 INFO:teuthology.orchestra.run.smithi106.stdout: thermald 2021-02-20T07:45:41.830 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2021-02-20T07:45:41.830 INFO:teuthology.orchestra.run.smithi106.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:45:41.830 INFO:teuthology.orchestra.run.smithi106.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T07:45:41.831 INFO:teuthology.orchestra.run.smithi106.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T07:45:41.831 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages will be upgraded: 2021-02-20T07:45:41.832 INFO:teuthology.orchestra.run.smithi106.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T07:45:42.006 INFO:teuthology.orchestra.run.smithi106.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:45:42.007 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 66.0 MB of archives. 2021-02-20T07:45:42.007 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T07:45:42.007 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T07:45:42.999 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T07:45:43.239 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T07:45:44.186 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T07:45:44.186 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T07:45:44.186 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T07:45:44.510 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T07:45:44.568 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T07:45:45.009 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 66.0 MB in 3s (24.2 MB/s) 2021-02-20T07:45:45.149 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T07:45:45.185 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T07:45:45.188 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:45:45.238 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:45:46.871 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T07:45:46.884 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:45:46.980 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:45:47.466 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T07:45:47.479 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:45:47.516 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:45:51.173 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:45:51.310 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:45:51.617 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:45:51.738 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:45:52.074 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T07:45:52.087 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T07:45:52.124 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:45:56.848 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T07:45:56.869 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:45:56.898 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:45:59.356 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:45:59.480 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:45:59.797 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:00.344 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:46:00.470 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:00.905 INFO:teuthology.orchestra.run.smithi106.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:46:00.905 INFO:teuthology.orchestra.run.smithi106.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T07:46:01.047 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:01.507 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:01.641 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T07:46:01.767 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T07:46:01.893 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T07:46:02.019 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:02.205 INFO:teuthology.orchestra.run.smithi106.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T07:46:02.205 INFO:teuthology.orchestra.run.smithi106.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:46:16.997 INFO:teuthology.orchestra.run.smithi106.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T07:46:16.998 INFO:teuthology.orchestra.run.smithi106.stdout:Sourcing file `/etc/default/grub' 2021-02-20T07:46:17.033 INFO:teuthology.orchestra.run.smithi106.stdout:Generating grub configuration file ... 2021-02-20T07:46:17.357 INFO:teuthology.orchestra.run.smithi106.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:46:17.398 INFO:teuthology.orchestra.run.smithi106.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:46:17.712 INFO:teuthology.orchestra.run.smithi106.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:46:17.742 INFO:teuthology.orchestra.run.smithi106.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:46:18.764 INFO:teuthology.orchestra.run.smithi106.stdout:done 2021-02-20T07:46:19.195 DEBUG:teuthology.orchestra.run.smithi106:> dpkg -s linux-image-generic 2021-02-20T07:46:19.214 INFO:teuthology.orchestra.run.smithi106.stdout:Package: linux-image-generic 2021-02-20T07:46:19.215 INFO:teuthology.orchestra.run.smithi106.stdout:Status: install ok installed 2021-02-20T07:46:19.215 INFO:teuthology.orchestra.run.smithi106.stdout:Priority: optional 2021-02-20T07:46:19.216 INFO:teuthology.orchestra.run.smithi106.stdout:Section: kernel 2021-02-20T07:46:19.216 INFO:teuthology.orchestra.run.smithi106.stdout:Installed-Size: 17 2021-02-20T07:46:19.216 INFO:teuthology.orchestra.run.smithi106.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:46:19.216 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: amd64 2021-02-20T07:46:19.217 INFO:teuthology.orchestra.run.smithi106.stdout:Source: linux-meta 2021-02-20T07:46:19.217 INFO:teuthology.orchestra.run.smithi106.stdout:Version: 4.15.0.135.122 2021-02-20T07:46:19.217 INFO:teuthology.orchestra.run.smithi106.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:46:19.217 INFO:teuthology.orchestra.run.smithi106.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:46:19.217 INFO:teuthology.orchestra.run.smithi106.stdout:Recommends: thermald 2021-02-20T07:46:19.218 INFO:teuthology.orchestra.run.smithi106.stdout:Description: Generic Linux kernel image 2021-02-20T07:46:19.218 INFO:teuthology.orchestra.run.smithi106.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:46:19.218 INFO:teuthology.orchestra.run.smithi106.stdout: available. 2021-02-20T07:46:19.219 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T07:46:19.219 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2021-02-20T07:46:19.225 INFO:teuthology.orchestra.run.smithi114.stdout:4.15.0-134-generic 2021-02-20T07:46:19.226 INFO:teuthology.task.kernel:Running kernel on smithi114: 4.15.0-134-generic 2021-02-20T07:46:19.226 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2021-02-20T07:46:19.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2021-02-20T07:46:19.564 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-02-20T07:46:19.686 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:46:19.766 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-02-20T07:46:19.828 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [916 kB] 2021-02-20T07:46:19.922 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,544 kB] 2021-02-20T07:46:19.948 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [298 kB] 2021-02-20T07:46:19.950 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [226 kB] 2021-02-20T07:46:19.952 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [29.9 kB] 2021-02-20T07:46:19.953 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,109 kB] 2021-02-20T07:46:19.964 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [975 kB] 2021-02-20T07:46:19.974 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [248 kB] 2021-02-20T07:46:20.040 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-02-20T07:46:20.156 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,220 kB] 2021-02-20T07:46:20.428 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,885 kB] 2021-02-20T07:46:20.531 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [390 kB] 2021-02-20T07:46:20.545 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [247 kB] 2021-02-20T07:46:20.554 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [33.4 kB] 2021-02-20T07:46:20.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,560 kB] 2021-02-20T07:46:20.591 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,718 kB] 2021-02-20T07:46:20.643 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [363 kB] 2021-02-20T07:46:20.653 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.3 kB] 2021-02-20T07:46:20.654 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.8 kB] 2021-02-20T07:46:20.654 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,300 B] 2021-02-20T07:46:21.725 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 13.1 MB in 2s (6,400 kB/s) 2021-02-20T07:46:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:46:23.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:46:23.257 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:46:23.514 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2021-02-20T07:46:23.514 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2021-02-20T07:46:23.593 INFO:teuthology.orchestra.run.smithi114.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:46:23.595 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:46:23.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:46:23.675 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:46:23.914 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2021-02-20T07:46:23.915 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2021-02-20T07:46:24.100 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2021-02-20T07:46:24.100 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:46:24.100 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-generic linux-image-4.15.0-135-generic 2021-02-20T07:46:24.101 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-4.15.0-135-generic linux-modules-extra-4.15.0-135-generic 2021-02-20T07:46:24.102 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2021-02-20T07:46:24.103 INFO:teuthology.orchestra.run.smithi114.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2021-02-20T07:46:24.103 INFO:teuthology.orchestra.run.smithi114.stdout:Recommended packages: 2021-02-20T07:46:24.103 INFO:teuthology.orchestra.run.smithi114.stdout: thermald 2021-02-20T07:46:24.139 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2021-02-20T07:46:24.140 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-4.15.0-135 linux-headers-4.15.0-135-generic 2021-02-20T07:46:24.140 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-4.15.0-135-generic linux-modules-4.15.0-135-generic 2021-02-20T07:46:24.142 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-extra-4.15.0-135-generic 2021-02-20T07:46:24.142 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages will be upgraded: 2021-02-20T07:46:24.143 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-generic linux-image-generic 2021-02-20T07:46:24.319 INFO:teuthology.orchestra.run.smithi114.stdout:3 upgraded, 5 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:46:24.320 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 66.0 MB of archives. 2021-02-20T07:46:24.320 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 334 MB of additional disk space will be used. 2021-02-20T07:46:24.320 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-135-generic amd64 4.15.0-135.139 [13.1 MB] 2021-02-20T07:46:25.320 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-135-generic amd64 4.15.0-135.139 [8,019 kB] 2021-02-20T07:46:25.564 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-135-generic amd64 4.15.0-135.139 [32.8 MB] 2021-02-20T07:46:26.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.135.122 [1,868 B] 2021-02-20T07:46:26.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.135.122 [2,592 B] 2021-02-20T07:46:26.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135 all 4.15.0-135.139 [10.9 MB] 2021-02-20T07:46:26.850 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-135-generic amd64 4.15.0-135.139 [1,097 kB] 2021-02-20T07:46:26.890 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.135.122 [2,488 B] 2021-02-20T07:46:27.344 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 66.0 MB in 3s (24.1 MB/s) 2021-02-20T07:46:27.499 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-4.15.0-135-generic. 2021-02-20T07:46:27.537 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102079 files and directories currently installed.) 2021-02-20T07:46:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../0-linux-modules-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:46:27.595 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:29.179 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-image-4.15.0-135-generic. 2021-02-20T07:46:29.190 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../1-linux-image-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:46:29.302 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:29.790 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-135-generic. 2021-02-20T07:46:29.802 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:46:29.840 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:33.475 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../3-linux-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:46:33.601 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:46:33.923 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:46:34.046 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:46:34.399 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-4.15.0-135. 2021-02-20T07:46:34.412 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../5-linux-headers-4.15.0-135_4.15.0-135.139_all.deb ... 2021-02-20T07:46:34.449 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:46:38.952 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-4.15.0-135-generic. 2021-02-20T07:46:38.973 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../6-linux-headers-4.15.0-135-generic_4.15.0-135.139_amd64.deb ... 2021-02-20T07:46:39.011 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:41.382 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.135.122_amd64.deb ... 2021-02-20T07:46:41.506 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-generic (4.15.0.135.122) over (4.15.0.134.121) ... 2021-02-20T07:46:42.357 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:42.894 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-4.15.0-135 (4.15.0-135.139) ... 2021-02-20T07:46:43.029 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:43.468 INFO:teuthology.orchestra.run.smithi114.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:46:43.468 INFO:teuthology.orchestra.run.smithi114.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-135-generic 2021-02-20T07:46:43.614 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-extra-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:44.107 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-generic (4.15.0.135.122) ... 2021-02-20T07:46:44.351 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-generic (4.15.0.135.122) ... 2021-02-20T07:46:44.477 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-generic (4.15.0.135.122) ... 2021-02-20T07:46:44.612 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for linux-image-4.15.0-135-generic (4.15.0-135.139) ... 2021-02-20T07:46:44.807 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-02-20T07:46:44.807 INFO:teuthology.orchestra.run.smithi114.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:46:59.297 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-02-20T07:46:59.298 INFO:teuthology.orchestra.run.smithi114.stdout:Sourcing file `/etc/default/grub' 2021-02-20T07:46:59.330 INFO:teuthology.orchestra.run.smithi114.stdout:Generating grub configuration file ... 2021-02-20T07:46:59.660 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:46:59.693 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:47:00.016 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:47:00.047 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:47:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:done 2021-02-20T07:47:01.528 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2021-02-20T07:47:01.551 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2021-02-20T07:47:01.552 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2021-02-20T07:47:01.552 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2021-02-20T07:47:01.552 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2021-02-20T07:47:01.552 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 17 2021-02-20T07:47:01.553 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:47:01.553 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2021-02-20T07:47:01.553 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2021-02-20T07:47:01.553 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 4.15.0.135.122 2021-02-20T07:47:01.554 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:47:01.554 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:47:01.554 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2021-02-20T07:47:01.554 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2021-02-20T07:47:01.554 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:47:01.555 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2021-02-20T07:47:01.555 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-134-generic Expected: 4.15.0-135-generic 2021-02-20T07:47:01.555 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-02-20T07:47:01.556 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T07:47:01.556 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-02-20T07:47:01.556 INFO:teuthology.task.kernel:Installing distro kernel on cluster1.mon.a... 2021-02-20T07:47:01.557 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T07:47:01.557 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get clean 2021-02-20T07:47:01.613 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2021-02-20T07:47:01.739 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:47:02.009 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:47:02.087 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:47:02.166 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:47:03.729 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:47:03.775 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:47:03.902 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:47:04.145 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2021-02-20T07:47:04.146 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2021-02-20T07:47:04.227 INFO:teuthology.orchestra.run.smithi106.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:47:04.228 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:47:04.229 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:47:04.304 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:47:04.534 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2021-02-20T07:47:04.535 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2021-02-20T07:47:04.734 INFO:teuthology.orchestra.run.smithi106.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T07:47:04.734 INFO:teuthology.orchestra.run.smithi106.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:47:04.736 DEBUG:teuthology.orchestra.run.smithi106:> dpkg -s linux-image-generic 2021-02-20T07:47:04.760 INFO:teuthology.orchestra.run.smithi106.stdout:Package: linux-image-generic 2021-02-20T07:47:04.760 INFO:teuthology.orchestra.run.smithi106.stdout:Status: install ok installed 2021-02-20T07:47:04.763 INFO:teuthology.orchestra.run.smithi106.stdout:Priority: optional 2021-02-20T07:47:04.764 INFO:teuthology.orchestra.run.smithi106.stdout:Section: kernel 2021-02-20T07:47:04.764 INFO:teuthology.orchestra.run.smithi106.stdout:Installed-Size: 17 2021-02-20T07:47:04.765 INFO:teuthology.orchestra.run.smithi106.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:47:04.765 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: amd64 2021-02-20T07:47:04.765 INFO:teuthology.orchestra.run.smithi106.stdout:Source: linux-meta 2021-02-20T07:47:04.766 INFO:teuthology.orchestra.run.smithi106.stdout:Version: 4.15.0.135.122 2021-02-20T07:47:04.766 INFO:teuthology.orchestra.run.smithi106.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:47:04.767 INFO:teuthology.orchestra.run.smithi106.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:47:04.767 INFO:teuthology.orchestra.run.smithi106.stdout:Recommends: thermald 2021-02-20T07:47:04.767 INFO:teuthology.orchestra.run.smithi106.stdout:Description: Generic Linux kernel image 2021-02-20T07:47:04.768 INFO:teuthology.orchestra.run.smithi106.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:47:04.769 INFO:teuthology.orchestra.run.smithi106.stdout: available. 2021-02-20T07:47:04.770 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2021-02-20T07:47:04.808 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.Bwuve3tYnV 2021-02-20T07:47:04.809 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Bwuve3tYnV 2021-02-20T07:47:04.865 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.Bwuve3tYnV 2021-02-20T07:47:04.988 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.Bwuve3tYnV is 8KB 2021-02-20T07:47:05.002 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.Bwuve3tYnV 2021-02-20T07:47:05.010 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T07:47:05.064 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:47:05.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T07:47:05.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T07:47:05.140 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T07:47:05.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo update-grub 2021-02-20T07:47:06.344 INFO:teuthology.orchestra.run.smithi106.stderr:Sourcing file `/etc/default/grub' 2021-02-20T07:47:06.368 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2021-02-20T07:47:06.603 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:47:06.616 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:47:06.925 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:47:06.937 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:47:07.550 INFO:teuthology.orchestra.run.smithi106.stderr:done 2021-02-20T07:47:07.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2021-02-20T07:47:07.554 INFO:teuthology.task.kernel:Installing distro kernel on cluster2.mon.a... 2021-02-20T07:47:07.554 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2021-02-20T07:47:07.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2021-02-20T07:47:07.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2021-02-20T07:47:07.764 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:47:07.875 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:47:07.953 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:47:08.039 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:47:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:47:09.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-02-20T07:47:09.761 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:47:10.007 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2021-02-20T07:47:10.008 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2021-02-20T07:47:10.089 INFO:teuthology.orchestra.run.smithi114.stderr:E: Unable to locate package linux-image-current-generic 2021-02-20T07:47:10.090 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-02-20T07:47:10.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-02-20T07:47:10.168 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:47:10.418 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2021-02-20T07:47:10.419 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2021-02-20T07:47:10.619 INFO:teuthology.orchestra.run.smithi114.stdout:linux-image-generic is already the newest version (4.15.0.135.122). 2021-02-20T07:47:10.619 INFO:teuthology.orchestra.run.smithi114.stdout:0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded. 2021-02-20T07:47:10.620 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2021-02-20T07:47:10.643 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2021-02-20T07:47:10.644 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2021-02-20T07:47:10.644 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2021-02-20T07:47:10.644 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2021-02-20T07:47:10.644 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 17 2021-02-20T07:47:10.645 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2021-02-20T07:47:10.645 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2021-02-20T07:47:10.645 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2021-02-20T07:47:10.645 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 4.15.0.135.122 2021-02-20T07:47:10.646 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-02-20T07:47:10.646 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-4.15.0-135-generic, linux-modules-extra-4.15.0-135-generic, linux-firmware, intel-microcode, amd64-microcode 2021-02-20T07:47:10.646 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2021-02-20T07:47:10.646 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2021-02-20T07:47:10.647 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2021-02-20T07:47:10.647 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2021-02-20T07:47:10.648 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2021-02-20T07:47:10.689 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.HgaReWVosf 2021-02-20T07:47:10.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HgaReWVosf 2021-02-20T07:47:10.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.HgaReWVosf 2021-02-20T07:47:10.858 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.HgaReWVosf is 8KB 2021-02-20T07:47:10.870 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.HgaReWVosf 2021-02-20T07:47:10.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-02-20T07:47:10.929 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:47:10.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-02-20T07:47:10.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-02-20T07:47:10.998 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-135-generic 2021-02-20T07:47:10.998 DEBUG:teuthology.orchestra.run.smithi114:> sudo update-grub 2021-02-20T07:47:12.209 INFO:teuthology.orchestra.run.smithi114.stderr:Sourcing file `/etc/default/grub' 2021-02-20T07:47:12.230 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2021-02-20T07:47:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-4.15.0-135-generic 2021-02-20T07:47:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-4.15.0-135-generic 2021-02-20T07:47:12.777 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-4.15.0-134-generic 2021-02-20T07:47:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-4.15.0-134-generic 2021-02-20T07:47:13.400 INFO:teuthology.orchestra.run.smithi114.stderr:done 2021-02-20T07:47:13.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2021-02-20T07:47:43.431 INFO:teuthology.misc:Re-opening connections... 2021-02-20T07:47:43.436 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:47:43.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:47:43.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:48:43.441 DEBUG:teuthology.orchestra.remote:timed out 2021-02-20T07:48:43.443 INFO:teuthology.misc:trying to connect to ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:48:43.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:48:43.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:48:46.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2021-02-20T07:48:46.504 DEBUG:teuthology.misc:waited 63.068352699279785 2021-02-20T07:48:47.505 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:48:47.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:48:47.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:48:50.771 DEBUG:teuthology.orchestra.run.smithi106:> true 2021-02-20T07:48:51.770 DEBUG:teuthology.misc:waited 68.33404064178467 2021-02-20T07:48:52.771 INFO:teuthology.misc:trying to connect to ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:48:52.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:48:52.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:48:54.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2021-02-20T07:48:54.819 DEBUG:teuthology.misc:waited 71.38336086273193 2021-02-20T07:48:55.820 INFO:teuthology.misc:trying to connect to ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:48:55.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:48:55.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:48:56.012 DEBUG:teuthology.orchestra.run.smithi114:> true 2021-02-20T07:48:57.310 DEBUG:teuthology.misc:waited 73.87451434135437 2021-02-20T07:48:58.312 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2021-02-20T07:48:58.312 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "4.15.0-135-generic"... 2021-02-20T07:48:58.313 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2021-02-20T07:48:58.319 INFO:teuthology.orchestra.run.smithi106.stdout:4.15.0-135-generic 2021-02-20T07:48:58.320 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T07:48:58.320 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T07:48:58.320 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T07:48:58.320 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2021-02-20T07:48:58.321 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "4.15.0-135-generic"... 2021-02-20T07:48:58.321 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2021-02-20T07:48:58.329 INFO:teuthology.orchestra.run.smithi114.stdout:4.15.0-135-generic 2021-02-20T07:48:58.331 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-135-generic vs 4.15.0-135-generic 2021-02-20T07:48:58.331 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-02-20T07:48:58.332 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-02-20T07:48:59.332 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2021-02-20T07:48:59.333 DEBUG:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T07:48:59.377 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2021-02-20T07:48:59.410 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2021-02-20T07:48:59.410 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-02-20T07:48:59.439 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2021-02-20T07:48:59.469 INFO:teuthology.run_tasks:Running task internal.base... 2021-02-20T07:48:59.478 INFO:teuthology.task.internal:Creating test directory... 2021-02-20T07:48:59.479 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T07:48:59.481 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-02-20T07:48:59.517 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-02-20T07:48:59.533 INFO:teuthology.run_tasks:Running task internal.archive... 2021-02-20T07:48:59.545 INFO:teuthology.task.internal:Creating archive directory... 2021-02-20T07:48:59.546 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T07:48:59.549 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-02-20T07:48:59.623 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-02-20T07:48:59.630 INFO:teuthology.task.internal:Enabling coredump saving... 2021-02-20T07:48:59.631 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T07:48:59.634 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-02-20T07:48:59.657 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:48:59.658 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:48:59.666 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:48:59.669 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-02-20T07:48:59.670 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-02-20T07:48:59.680 INFO:teuthology.task.internal:Configuring sudo... 2021-02-20T07:48:59.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T07:48:59.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-02-20T07:48:59.728 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-02-20T07:48:59.738 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-02-20T07:48:59.739 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T07:48:59.765 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-02-20T07:48:59.776 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T07:48:59.820 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T07:48:59.866 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:48:59.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T07:48:59.925 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-02-20T07:48:59.931 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-02-20T07:48:59.980 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:48:59.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-02-20T07:49:00.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2021-02-20T07:49:00.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2021-02-20T07:49:00.181 INFO:teuthology.run_tasks:Running task internal.timer... 2021-02-20T07:49:00.191 INFO:teuthology.task.internal:Starting timer... 2021-02-20T07:49:00.191 INFO:teuthology.run_tasks:Running task pcp... 2021-02-20T07:49:00.213 INFO:teuthology.run_tasks:Running task selinux... 2021-02-20T07:49:00.227 DEBUG:teuthology.task.selinux:Excluding smithi106: OS 'ubuntu' does not support SELinux 2021-02-20T07:49:00.227 DEBUG:teuthology.task.selinux:Excluding smithi114: OS 'ubuntu' does not support SELinux 2021-02-20T07:49:00.228 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-02-20T07:49:00.228 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-02-20T07:49:00.228 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-02-20T07:49:00.229 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-02-20T07:49:00.242 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-02-20T07:49:00.243 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-02-20T07:49:00.260 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-02-20T07:49:00.263 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-02-20T07:49:00.619 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-02-20T07:49:01.889 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-02-20T07:49:04.561 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-02-20T07:49:04.583 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:04 +0000 (0:00:00.079) 0:00:00.079 ***** 2021-02-20T07:49:04.597 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:04 +0000 (0:00:00.013) 0:00:00.093 ***** 2021-02-20T07:49:06.364 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi106.front.sepia.ceph.com] 2021-02-20T07:49:06.446 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:06 +0000 (0:00:01.849) 0:00:01.942 ***** 2021-02-20T07:49:06.456 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] 2021-02-20T07:49:06.541 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:06 +0000 (0:00:00.089) 0:00:02.032 ***** 2021-02-20T07:49:06.733 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:49:06.777 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:06 +0000 (0:00:00.240) 0:00:02.273 ***** 2021-02-20T07:49:06.784 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:49:06.822 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:06 +0000 (0:00:00.044) 0:00:02.317 ***** 2021-02-20T07:49:07.295 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** changed: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T07:49:07.339 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:07 +0000 (0:00:00.517) 0:00:02.834 ***** 2021-02-20T07:49:07.350 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-02-20T07:49:07.391 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:07 +0000 (0:00:00.052) 0:00:02.887 ***** 2021-02-20T07:49:07.704 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-02-20T07:49:07.705 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.077331", "end": "2021-02-20 07:49:07.700398", "rc": 0, "start": "2021-02-20 07:49:07.623067", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.074614", "end": "2021-02-20 07:49:07.734973", "rc": 0, "start": "2021-02-20 07:49:07.660359", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-02-20T07:49:07.747 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:07 +0000 (0:00:00.355) 0:00:03.243 ***** 2021-02-20T07:49:07.781 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:07 +0000 (0:00:00.033) 0:00:03.277 ***** 2021-02-20T07:49:08.005 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-02-20T07:49:08.007 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:49:08.056 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:08 +0000 (0:00:00.275) 0:00:03.552 ***** 2021-02-20T07:49:08.092 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:08 +0000 (0:00:00.035) 0:00:03.588 ***** 2021-02-20T07:49:08.769 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-02-20T07:49:08.771 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807348.0838053-188776581627367/source", "state": "file", "uid": 0} 2021-02-20T07:49:08.812 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:08 +0000 (0:00:00.719) 0:00:04.308 ***** 2021-02-20T07:49:08.821 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807348.1133082-2947358387433/source", "state": "file", "uid": 0} 2021-02-20T07:49:08.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:08 +0000 (0:00:00.051) 0:00:04.359 ***** 2021-02-20T07:49:09.152 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-02-20T07:49:09.154 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T07:49:09.156 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T07:49:09.196 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-02-20T07:49:09.198 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-02-20T07:49:09.318 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-02-20T07:49:09.326 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-02-20T07:49:09.511 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:09 +0000 (0:00:00.647) 0:00:05.007 ***** 2021-02-20T07:49:09.546 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:09 +0000 (0:00:00.034) 0:00:05.042 ***** 2021-02-20T07:49:10.275 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi114.front.sepia.ceph.com] 2021-02-20T07:49:10.378 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Saturday 20 February 2021 07:49:10 +0000 (0:00:00.832) 0:00:05.874 ***** 2021-02-20T07:49:10.744 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-20T07:49:10.746 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.160952", "end": "2021-02-20 07:49:10.710947", "rc": 0, "start": "2021-02-20 07:49:10.549995", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T07:49:10.782 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:10 +0000 (0:00:00.403) 0:00:06.278 ***** 2021-02-20T07:49:10.944 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.015818", "end": "2021-02-20 07:49:10.945406", "failed_when_result": false, "rc": 0, "start": "2021-02-20 07:49:10.929588", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T07:49:10.982 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:10 +0000 (0:00:00.200) 0:00:06.479 ***** 2021-02-20T07:49:11.013 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-20T07:49:11.015 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:11.052 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.069) 0:00:06.548 ***** 2021-02-20T07:49:11.079 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-02-20T07:49:11.081 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:11.115 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.063) 0:00:06.611 ***** 2021-02-20T07:49:11.142 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:11.180 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.064) 0:00:06.676 ***** 2021-02-20T07:49:11.218 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2021-02-20T07:49:11.219 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:11.257 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.077) 0:00:06.753 ***** 2021-02-20T07:49:11.274 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi106.front.sepia.ceph.com] 2021-02-20T07:49:11.316 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:11.359 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.101) 0:00:06.855 ***** 2021-02-20T07:49:11.394 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.034) 0:00:06.890 ***** 2021-02-20T07:49:11.536 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-02-20T07:49:11.538 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:49:11.574 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.179) 0:00:07.070 ***** 2021-02-20T07:49:11.797 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T07:49:11.799 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T07:49:11.898 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-02-20T07:49:11.900 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.225676", "end": "2021-02-20 07:49:11.837475", "rc": 0, "start": "2021-02-20 07:49:11.611799", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-02-20T07:49:11.969 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:11 +0000 (0:00:00.394) 0:00:07.465 ***** 2021-02-20T07:49:11.981 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T07:49:11.983 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T07:49:12.104 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T07:49:12.139 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.028763", "end": "2021-02-20 07:49:12.166408", "failed_when_result": false, "rc": 0, "start": "2021-02-20 07:49:12.137645", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-02-20T07:49:12.180 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.210) 0:00:07.676 ***** 2021-02-20T07:49:12.207 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-02-20T07:49:12.209 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:12.244 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.064) 0:00:07.740 ***** 2021-02-20T07:49:12.265 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T07:49:12.287 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:12.326 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.081) 0:00:07.822 ***** 2021-02-20T07:49:12.355 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:12.392 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.066) 0:00:07.889 ***** 2021-02-20T07:49:12.400 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T07:49:12.402 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T07:49:12.433 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:12.471 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.077) 0:00:07.966 ***** 2021-02-20T07:49:12.498 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:12.536 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.064) 0:00:08.031 ***** 2021-02-20T07:49:12.542 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T07:49:12.546 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T07:49:12.659 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T07:49:12.691 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-02-20T07:49:12.693 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:49:12.735 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:12 +0000 (0:00:00.200) 0:00:08.231 ***** 2021-02-20T07:49:12.784 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T07:49:12.947 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T07:49:12.958 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-02-20T07:49:13.097 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-02-20T07:49:13.098 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T07:49:13.239 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-02-20T07:49:13.241 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T07:49:13.384 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-02-20T07:49:13.397 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T07:49:13.525 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-02-20T07:49:13.548 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T07:49:13.665 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-02-20T07:49:13.687 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T07:49:13.785 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-02-20T07:49:13.818 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T07:49:13.925 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-02-20T07:49:13.957 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T07:49:14.054 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-02-20T07:49:14.088 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T07:49:14.195 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-02-20T07:49:14.238 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T07:49:14.333 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-02-20T07:49:14.386 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T07:49:14.460 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-02-20T07:49:14.525 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T07:49:14.579 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-02-20T07:49:14.677 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T07:49:14.721 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-02-20T07:49:14.827 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T07:49:14.860 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-02-20T07:49:14.975 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T07:49:14.987 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-02-20T07:49:15.125 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T07:49:15.126 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-02-20T07:49:15.264 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-02-20T07:49:15.265 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T07:49:15.382 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T07:49:15.393 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-02-20T07:49:15.510 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-02-20T07:49:15.522 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T07:49:15.639 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-02-20T07:49:15.662 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T07:49:15.789 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-02-20T07:49:15.811 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T07:49:15.918 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-02-20T07:49:15.950 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T07:49:16.057 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-02-20T07:49:16.100 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T07:49:16.176 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-02-20T07:49:16.252 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T07:49:16.307 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-02-20T07:49:16.382 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T07:49:16.446 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-02-20T07:49:16.521 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T07:49:16.565 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-02-20T07:49:16.685 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T07:49:16.707 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-02-20T07:49:16.709 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:16.879 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-02-20T07:49:16.918 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T07:49:17.039 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-02-20T07:49:17.100 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T07:49:17.164 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-02-20T07:49:17.249 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T07:49:17.303 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-02-20T07:49:17.392 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T07:49:17.436 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-02-20T07:49:17.534 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T07:49:17.567 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-02-20T07:49:17.663 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T07:49:17.716 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-02-20T07:49:17.813 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T07:49:17.857 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-02-20T07:49:17.955 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T07:49:17.998 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-02-20T07:49:18.095 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T07:49:18.139 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T07:49:18.247 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T07:49:18.270 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-02-20T07:49:18.377 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T07:49:18.410 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-02-20T07:49:18.527 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T07:49:18.538 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-02-20T07:49:18.666 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T07:49:18.667 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-02-20T07:49:18.794 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-02-20T07:49:18.816 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T07:49:18.921 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-02-20T07:49:18.964 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T07:49:19.060 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-02-20T07:49:19.113 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T07:49:19.188 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T07:49:19.246 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T07:49:19.333 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-02-20T07:49:19.387 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T07:49:19.463 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-02-20T07:49:19.528 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T07:49:19.583 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-02-20T07:49:19.683 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T07:49:19.706 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-02-20T07:49:19.823 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T07:49:19.845 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-02-20T07:49:19.963 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T07:49:19.977 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-02-20T07:49:20.101 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-02-20T07:49:20.113 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T07:49:20.241 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-02-20T07:49:20.253 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T07:49:20.392 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T07:49:20.394 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-02-20T07:49:20.533 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-02-20T07:49:20.535 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T07:49:20.674 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-02-20T07:49:20.697 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T07:49:20.804 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-02-20T07:49:20.837 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T07:49:20.933 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-02-20T07:49:20.971 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T07:49:21.056 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-02-20T07:49:21.121 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T07:49:21.207 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-02-20T07:49:21.261 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T07:49:21.357 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-02-20T07:49:21.390 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T07:49:21.507 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-02-20T07:49:21.530 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T07:49:21.658 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-02-20T07:49:21.670 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T07:49:21.799 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T07:49:21.810 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-02-20T07:49:21.937 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T07:49:21.939 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-02-20T07:49:22.066 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T07:49:22.088 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-02-20T07:49:22.205 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T07:49:22.217 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-02-20T07:49:22.334 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T07:49:22.345 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-02-20T07:49:22.481 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T07:49:22.493 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-02-20T07:49:22.619 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T07:49:22.641 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-02-20T07:49:22.757 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T07:49:22.790 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-02-20T07:49:22.896 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T07:49:22.929 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-02-20T07:49:23.034 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T07:49:23.066 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-02-20T07:49:23.176 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T07:49:23.219 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-02-20T07:49:23.306 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T07:49:23.383 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-02-20T07:49:23.459 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T07:49:23.524 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-02-20T07:49:23.590 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T07:49:23.676 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-02-20T07:49:23.742 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T07:49:23.817 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-02-20T07:49:23.884 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T07:49:23.971 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-02-20T07:49:24.035 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T07:49:24.121 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-02-20T07:49:24.186 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T07:49:24.284 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-02-20T07:49:24.339 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T07:49:24.406 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-02-20T07:49:24.486 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T07:49:24.563 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-02-20T07:49:24.641 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T07:49:24.727 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-02-20T07:49:24.792 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T07:49:24.878 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-02-20T07:49:24.921 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T07:49:25.006 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-02-20T07:49:25.039 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T07:49:25.161 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-02-20T07:49:25.183 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T07:49:25.292 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-02-20T07:49:25.325 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T07:49:25.440 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-02-20T07:49:25.462 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T07:49:25.577 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-02-20T07:49:25.589 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T07:49:25.707 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-02-20T07:49:25.718 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T07:49:25.856 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-02-20T07:49:25.868 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T07:49:26.005 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-02-20T07:49:26.017 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T07:49:26.133 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-02-20T07:49:26.155 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T07:49:26.284 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T07:49:26.307 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-02-20T07:49:26.413 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T07:49:26.435 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-02-20T07:49:26.560 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T07:49:26.571 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-02-20T07:49:26.707 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-02-20T07:49:26.708 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T07:49:26.844 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-02-20T07:49:26.856 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T07:49:26.984 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-02-20T07:49:27.006 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T07:49:27.145 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-02-20T07:49:27.157 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T07:49:27.306 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-02-20T07:49:27.339 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T07:49:27.452 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-02-20T07:49:27.518 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T07:49:27.620 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-02-20T07:49:27.669 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T07:49:27.760 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-02-20T07:49:27.805 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T07:49:27.900 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-02-20T07:49:27.954 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T07:49:28.042 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-02-20T07:49:28.120 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T07:49:28.184 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-02-20T07:49:28.269 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T07:49:28.323 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-02-20T07:49:28.421 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T07:49:28.466 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-02-20T07:49:28.562 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T07:49:28.595 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-02-20T07:49:28.703 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T07:49:28.727 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-02-20T07:49:28.857 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T07:49:28.880 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-02-20T07:49:29.019 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T07:49:29.021 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-02-20T07:49:29.161 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-02-20T07:49:29.162 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T07:49:29.290 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-02-20T07:49:29.302 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T07:49:29.439 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-02-20T07:49:29.451 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T07:49:29.579 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-02-20T07:49:29.612 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T07:49:29.729 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-02-20T07:49:29.762 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T07:49:29.890 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-02-20T07:49:29.912 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T07:49:30.043 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-02-20T07:49:30.045 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T07:49:30.166 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-02-20T07:49:30.199 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T07:49:30.315 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-02-20T07:49:30.348 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T07:49:30.445 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-02-20T07:49:30.500 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T07:49:30.596 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-02-20T07:49:30.652 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T07:49:30.739 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-02-20T07:49:30.793 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T07:49:30.903 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-02-20T07:49:30.926 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T07:49:31.056 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-02-20T07:49:31.079 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T07:49:31.184 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-02-20T07:49:31.242 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T07:49:31.340 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-02-20T07:49:31.373 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T07:49:31.491 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-02-20T07:49:31.503 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T07:49:31.626 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-02-20T07:49:31.648 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T07:49:31.757 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-02-20T07:49:31.911 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-02-20T07:49:32.049 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-02-20T07:49:32.135 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T07:49:32.227 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-02-20T07:49:32.268 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:32 +0000 (0:00:19.532) 0:00:27.764 ***** 2021-02-20T07:49:32.298 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-20T07:49:32.334 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:32 +0000 (0:00:00.065) 0:00:27.830 ***** 2021-02-20T07:49:32.386 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T07:49:32.389 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T07:49:32.391 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T07:49:32.393 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T07:49:32.394 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T07:49:32.396 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T07:49:32.397 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T07:49:32.399 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 2021-02-20T07:49:32.400 INFO:teuthology.task.ansible.out:5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T07:49:32.401 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveirada 2021-02-20T07:49:32.403 INFO:teuthology.task.ansible.out:n@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOe 2021-02-20T07:49:32.404 INFO:teuthology.task.ansible.out:nVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9 2021-02-20T07:49:32.406 INFO:teuthology.task.ansible.out:gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": 2021-02-20T07:49:32.408 INFO:teuthology.task.ansible.out:"dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NI 2021-02-20T07:49:32.410 INFO:teuthology.task.ansible.out:Nj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395 2021-02-20T07:49:32.410 INFO:teuthology.task.ansible.out:862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T07:49:32.459 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:32 +0000 (0:00:00.124) 0:00:27.955 ***** 2021-02-20T07:49:32.879 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-02-20T07:49:32.881 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-02-20T07:49:33.029 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:33 +0000 (0:00:00.570) 0:00:28.525 ***** 2021-02-20T07:49:33.068 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-02-20T07:49:33.108 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:33 +0000 (0:00:00.078) 0:00:28.604 ***** 2021-02-20T07:49:33.173 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-02-20T07:49:33.176 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-02-20T07:49:33.178 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-02-20T07:49:33.180 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-02-20T07:49:33.181 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1Wi 2021-02-20T07:49:33.183 INFO:teuthology.task.ansible.out:RenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vum 2021-02-20T07:49:33.185 INFO:teuthology.task.ansible.out:rao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69 2021-02-20T07:49:33.186 INFO:teuthology.task.ansible.out:mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 2021-02-20T07:49:33.187 INFO:teuthology.task.ansible.out:5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d8 2021-02-20T07:49:33.188 INFO:teuthology.task.ansible.out:41a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveirada 2021-02-20T07:49:33.190 INFO:teuthology.task.ansible.out:n@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOe 2021-02-20T07:49:33.192 INFO:teuthology.task.ansible.out:nVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9 2021-02-20T07:49:33.193 INFO:teuthology.task.ansible.out:gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": 2021-02-20T07:49:33.194 INFO:teuthology.task.ansible.out:"mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": 2021-02-20T07:49:33.196 INFO:teuthology.task.ansible.out:"dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NI 2021-02-20T07:49:33.197 INFO:teuthology.task.ansible.out:Nj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-02-20T07:49:33.242 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:33 +0000 (0:00:00.133) 0:00:28.738 ***** 2021-02-20T07:49:33.663 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-02-20T07:49:33.664 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T07:49:33.700 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:33 +0000 (0:00:00.458) 0:00:29.196 ***** 2021-02-20T07:49:33.846 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com -> localhost] => {"after": "df8946f08e870e2d2565b79099289ca48bd825f3", "attempts": 1, "before": "df8946f08e870e2d2565b79099289ca48bd825f3", "changed": false, "remote_url_changed": false} 2021-02-20T07:49:33.883 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:33 +0000 (0:00:00.183) 0:00:29.379 ***** 2021-02-20T07:49:33.902 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T07:49:33.903 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T07:49:34.051 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T07:49:34.063 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-02-20T07:49:34.200 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T07:49:34.202 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-02-20T07:49:34.326 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-02-20T07:49:34.337 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-02-20T07:49:34.339 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-20T07:49:34.453 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-02-20T07:49:34.455 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-02-20T07:49:34.486 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T07:49:34.581 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-02-20T07:49:34.613 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T07:49:34.615 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-20T07:49:34.718 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-02-20T07:49:34.720 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-02-20T07:49:34.761 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T07:49:34.857 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-02-20T07:49:34.900 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T07:49:34.994 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-02-20T07:49:35.048 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T07:49:35.059 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.061 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:35.123 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-02-20T07:49:35.135 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.191 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T07:49:35.194 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T07:49:35.276 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-02-20T07:49:35.277 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-02-20T07:49:35.328 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T07:49:35.330 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-02-20T07:49:35.339 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.350 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.407 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-02-20T07:49:35.409 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-02-20T07:49:35.418 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.429 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:35.484 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T07:49:35.568 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-02-20T07:49:35.622 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T07:49:35.624 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T07:49:35.696 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-02-20T07:49:35.698 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-02-20T07:49:35.762 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T07:49:35.836 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-02-20T07:49:35.900 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T07:49:35.985 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-02-20T07:49:36.060 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T07:49:36.124 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-02-20T07:49:36.199 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T07:49:36.254 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-02-20T07:49:36.328 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T07:49:36.393 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-02-20T07:49:36.467 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T07:49:36.469 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T07:49:36.542 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-02-20T07:49:36.544 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-02-20T07:49:36.606 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T07:49:36.608 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T07:49:36.671 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-02-20T07:49:36.673 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-02-20T07:49:36.735 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T07:49:36.809 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-02-20T07:49:36.874 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T07:49:36.949 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-02-20T07:49:37.023 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T07:49:37.024 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T07:49:37.098 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-02-20T07:49:37.100 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-02-20T07:49:37.151 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T07:49:37.226 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-02-20T07:49:37.269 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T07:49:37.375 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-02-20T07:49:37.397 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T07:49:37.525 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-02-20T07:49:37.536 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T07:49:37.674 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-02-20T07:49:37.685 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T07:49:37.812 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-02-20T07:49:37.824 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T07:49:37.826 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T07:49:37.952 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-02-20T07:49:37.953 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-02-20T07:49:37.963 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T07:49:38.100 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-02-20T07:49:38.112 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T07:49:38.248 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-02-20T07:49:38.260 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T07:49:38.387 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-02-20T07:49:38.388 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T07:49:38.505 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-02-20T07:49:38.527 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T07:49:38.623 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-02-20T07:49:38.676 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T07:49:38.781 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-02-20T07:49:38.792 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T07:49:38.910 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T07:49:38.942 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T07:49:39.026 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-02-20T07:49:39.090 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T07:49:39.164 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-02-20T07:49:39.230 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T07:49:39.232 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T07:49:39.305 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-02-20T07:49:39.307 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-02-20T07:49:39.379 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-02-20T07:49:39.381 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-20T07:49:39.421 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-02-20T07:49:39.423 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-02-20T07:49:39.517 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T07:49:39.549 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-02-20T07:49:39.657 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T07:49:39.679 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-02-20T07:49:39.798 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T07:49:39.811 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-02-20T07:49:39.939 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T07:49:39.941 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T07:49:39.962 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-02-20T07:49:39.963 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T07:49:40.074 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T07:49:40.076 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T07:49:40.078 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T07:49:40.087 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-02-20T07:49:40.089 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-02-20T07:49:40.091 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-02-20T07:49:40.228 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T07:49:40.230 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-02-20T07:49:40.230 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:40.402 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T07:49:40.407 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-02-20T07:49:40.570 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T07:49:40.582 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-02-20T07:49:40.724 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T07:49:40.725 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-02-20T07:49:40.862 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T07:49:40.874 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-02-20T07:49:41.011 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-02-20T07:49:41.013 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0b 2021-02-20T07:49:41.015 INFO:teuthology.task.ansible.out:sfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-02-20T07:49:41.151 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T07:49:41.153 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-02-20T07:49:41.273 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T07:49:41.296 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-02-20T07:49:41.403 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T07:49:41.404 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T07:49:41.457 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-02-20T07:49:41.459 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-02-20T07:49:41.522 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T07:49:41.608 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-02-20T07:49:41.674 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T07:49:41.765 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-02-20T07:49:41.820 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T07:49:41.822 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T07:49:41.936 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-02-20T07:49:41.938 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-02-20T07:49:41.959 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T07:49:42.066 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-02-20T07:49:42.109 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T07:49:42.228 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-02-20T07:49:42.297 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T07:49:42.374 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-02-20T07:49:42.462 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T07:49:42.559 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-02-20T07:49:42.624 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T07:49:42.731 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-02-20T07:49:42.767 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T07:49:42.853 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-02-20T07:49:42.917 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T07:49:43.013 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-02-20T07:49:43.046 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T07:49:43.153 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-02-20T07:49:43.176 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T07:49:43.309 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-02-20T07:49:43.311 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-02-20T07:49:43.469 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T07:49:43.611 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T07:49:43.622 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-02-20T07:49:43.625 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:43.760 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-02-20T07:49:43.762 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T07:49:43.901 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-02-20T07:49:43.923 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-02-20T07:49:44.063 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROh 2021-02-20T07:49:44.064 INFO:teuthology.task.ansible.out:wQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T07:49:44.189 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T07:49:44.201 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-02-20T07:49:44.319 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T07:49:44.330 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-02-20T07:49:44.457 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-02-20T07:49:44.471 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T07:49:44.609 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T07:49:44.610 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-02-20T07:49:44.751 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-02-20T07:49:44.754 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T07:49:44.889 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-02-20T07:49:44.922 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-02-20T07:49:44.924 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-02-20T07:49:45.039 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-02-20T07:49:45.041 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-20T07:49:45.185 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-02-20T07:49:45.187 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T07:49:45.336 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-20T07:49:45.347 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-02-20T07:49:45.465 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-20T07:49:45.487 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-02-20T07:49:45.605 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-02-20T07:49:45.607 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-20T07:49:45.616 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-02-20T07:49:45.754 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-02-20T07:49:45.756 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-20T07:49:45.765 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-02-20T07:49:45.767 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-02-20T07:49:45.888 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-20T07:49:45.910 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-02-20T07:49:45.911 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-02-20T07:49:46.027 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T07:49:46.051 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-02-20T07:49:46.054 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:46.178 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T07:49:46.200 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-02-20T07:49:46.309 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T07:49:46.342 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-02-20T07:49:46.471 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-02-20T07:49:46.484 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T07:49:46.591 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-02-20T07:49:46.624 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T07:49:46.732 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-02-20T07:49:46.776 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T07:49:46.878 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-02-20T07:49:46.921 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-02-20T07:49:46.923 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T07:49:47.029 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-02-20T07:49:47.032 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-02-20T07:49:47.062 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-20T07:49:47.189 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-02-20T07:49:47.191 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-20T07:49:47.319 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T07:49:47.342 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-02-20T07:49:47.437 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T07:49:47.490 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-02-20T07:49:47.587 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T07:49:47.640 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-02-20T07:49:47.715 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T07:49:47.791 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-02-20T07:49:47.844 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T07:49:47.846 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T07:49:47.909 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-02-20T07:49:47.962 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-20T07:49:48.047 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-02-20T07:49:48.049 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-02-20T07:49:48.101 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T07:49:48.253 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-02-20T07:49:48.258 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-20T07:49:48.260 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:48.379 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-02-20T07:49:48.437 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T07:49:48.522 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-02-20T07:49:48.575 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T07:49:48.663 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-02-20T07:49:48.728 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T07:49:48.792 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-02-20T07:49:48.793 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:48.858 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T07:49:48.942 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-02-20T07:49:48.985 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T07:49:49.081 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-02-20T07:49:49.113 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T07:49:49.208 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-02-20T07:49:49.251 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T07:49:49.351 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-02-20T07:49:49.405 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T07:49:49.480 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-02-20T07:49:49.533 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T07:49:49.628 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-02-20T07:49:49.661 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T07:49:49.767 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-02-20T07:49:49.799 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T07:49:49.801 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:49.896 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-02-20T07:49:49.918 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T07:49:50.035 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T07:49:50.058 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-02-20T07:49:50.059 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:50.144 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T07:49:50.166 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-02-20T07:49:50.283 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T07:49:50.315 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-02-20T07:49:50.432 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T07:49:50.434 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T07:49:50.454 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-02-20T07:49:50.570 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T07:49:50.572 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-02-20T07:49:50.709 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T07:49:50.721 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-02-20T07:49:50.724 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-02-20T07:49:50.850 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T07:49:50.863 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-02-20T07:49:51.003 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T07:49:51.038 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-02-20T07:49:51.135 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T07:49:51.169 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-02-20T07:49:51.279 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-20T07:49:51.300 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-02-20T07:49:51.413 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T07:49:51.457 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-02-20T07:49:51.541 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T07:49:51.543 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:51.605 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-02-20T07:49:51.607 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:51.681 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T07:49:51.756 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-02-20T07:49:51.809 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T07:49:51.862 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-02-20T07:49:51.937 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-20T07:49:52.021 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-02-20T07:49:52.054 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T07:49:52.196 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-02-20T07:49:52.208 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T07:49:52.313 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-02-20T07:49:52.346 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T07:49:52.442 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-02-20T07:49:52.464 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T07:49:52.591 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-02-20T07:49:52.593 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-02-20T07:49:52.718 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T07:49:52.740 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-02-20T07:49:52.864 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T07:49:52.912 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-02-20T07:49:52.950 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:52 +0000 (0:00:19.067) 0:00:48.446 ***** 2021-02-20T07:49:53.002 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T07:49:53.004 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.014 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.016 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.026 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T07:49:53.028 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c 2021-02-20T07:49:53.029 INFO:teuthology.task.ansible.out:790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.038 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.050 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.061 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.072 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.074 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.137 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-02-20T07:49:53.139 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-02-20T07:49:53.224 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T07:49:53.226 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T07:49:53.236 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.247 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.290 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi114.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-02-20T07:49:53.392 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:53 +0000 (0:00:00.442) 0:00:48.888 ***** 2021-02-20T07:49:53.403 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-02-20T07:49:53.405 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T07:49:53.439 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.450 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.461 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.473 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.475 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.495 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.506 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.516 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.527 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.529 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T07:49:53.541 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.552 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.563 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.565 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.575 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.586 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.597 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.608 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.620 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.631 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.642 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.653 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.656 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.664 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.676 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-02-20T07:49:53.677 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.687 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.688 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.699 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.710 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.721 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.732 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.744 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.753 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.765 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.776 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.787 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.789 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.799 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.809 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.821 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.833 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.844 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.848 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-02-20T07:49:53.851 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.851 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.863 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.876 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.878 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.889 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.901 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.903 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.913 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.913 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.927 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.938 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.939 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.950 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.963 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.972 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.975 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} ok: [smithi114.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+ 2021-02-20T07:49:53.977 INFO:teuthology.task.ansible.out:i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-02-20T07:49:53.985 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.987 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.996 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:53.998 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.008 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.010 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.020 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.022 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.036 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.047 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.049 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.058 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.060 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.070 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.083 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": " 2021-02-20T07:49:54.085 INFO:teuthology.task.ansible.out:dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.095 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.105 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.106 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.117 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.119 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.130 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.131 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.142 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.144 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.155 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.156 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.156 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.167 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.169 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.180 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.181 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.191 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.194 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.204 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.206 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.216 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.218 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.229 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.231 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.240 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.242 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.243 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:54.254 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.256 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.266 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.268 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.278 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.280 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.291 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.293 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.303 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.315 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.317 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.331 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.333 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.344 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.345 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.355 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.357 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.367 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.369 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.379 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.381 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.392 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.393 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.404 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.406 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.406 INFO:teuthology.task.ansible.out: 2021-02-20T07:49:54.407 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.418 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.419 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.430 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.432 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.442 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.444 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.454 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.456 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.466 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.468 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.479 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.481 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.492 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.493 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.495 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.505 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.507 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.517 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.519 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.530 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.531 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.541 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.543 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.554 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.556 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.566 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.568 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.578 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.579 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.579 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.590 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.592 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.615 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS 2021-02-20T07:49:54.618 INFO:teuthology.task.ansible.out:0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.626 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.628 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.639 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.641 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.650 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.652 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.668 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.706 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:54 +0000 (0:00:01.313) 0:00:50.202 ***** 2021-02-20T07:49:54.714 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.715 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.716 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.718 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ra 2021-02-20T07:49:54.719 INFO:teuthology.task.ansible.out:kesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.731 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.733 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.735 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.774 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:54 +0000 (0:00:00.068) 0:00:50.270 ***** 2021-02-20T07:49:54.784 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.786 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.796 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.799 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.808 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.811 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.821 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.824 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.833 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.844 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.855 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.857 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.867 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.878 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.880 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.890 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.904 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.912 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.914 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.924 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.936 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.938 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.949 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.961 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.963 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.972 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T07:49:54.975 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.985 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.996 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:54.999 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.008 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.019 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.020 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.032 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.098 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:55 +0000 (0:00:00.324) 0:00:50.594 ***** 2021-02-20T07:49:55.105 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T07:49:55.107 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T07:49:55.149 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-02-20T07:49:55.151 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:49:55.186 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:49:55 +0000 (0:00:00.087) 0:00:50.682 ***** 2021-02-20T07:49:55.223 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T07:49:55.225 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T07:49:55.341 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-02-20T07:49:55.352 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-20T07:49:55.480 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T07:49:55.482 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-02-20T07:49:55.608 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-02-20T07:49:55.609 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-20T07:49:55.717 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T07:49:55.719 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-02-20T07:49:55.848 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-02-20T07:49:55.870 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T07:49:55.990 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-02-20T07:49:56.018 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T07:49:56.125 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-02-20T07:49:56.159 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T07:49:56.248 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-02-20T07:49:56.292 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-20T07:49:56.370 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-02-20T07:49:56.415 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T07:49:56.543 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-02-20T07:49:56.555 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T07:49:56.675 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-02-20T07:49:56.719 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T07:49:56.795 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-02-20T07:49:56.827 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T07:49:56.944 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-02-20T07:49:56.966 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-20T07:49:57.072 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-02-20T07:49:57.116 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T07:49:57.200 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-02-20T07:49:57.245 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T07:49:57.310 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-02-20T07:49:57.386 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T07:49:57.430 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-02-20T07:49:57.527 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T07:49:57.559 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-02-20T07:49:57.655 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T07:49:57.687 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-02-20T07:49:57.794 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T07:49:57.838 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-02-20T07:49:57.934 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T07:49:57.966 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-02-20T07:49:58.083 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T07:49:58.105 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-02-20T07:49:58.203 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T07:49:58.236 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-02-20T07:49:58.331 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T07:49:58.376 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-02-20T07:49:58.460 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T07:49:58.514 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-02-20T07:49:58.589 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T07:49:58.656 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-02-20T07:49:58.731 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T07:49:58.775 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-02-20T07:49:58.851 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T07:49:58.894 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-02-20T07:49:58.991 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T07:49:59.036 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-02-20T07:49:59.132 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T07:49:59.166 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-02-20T07:49:59.272 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T07:49:59.315 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-02-20T07:49:59.396 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T07:49:59.449 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-02-20T07:49:59.531 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T07:49:59.594 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-02-20T07:49:59.678 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T07:49:59.731 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-02-20T07:49:59.805 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T07:49:59.857 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-02-20T07:49:59.931 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T07:50:00.015 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-02-20T07:50:00.134 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-02-20T07:50:00.283 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-02-20T07:50:00.327 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T07:50:00.446 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-20T07:50:00.447 INFO:teuthology.task.ansible.out: 2021-02-20T07:50:00.564 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T07:50:00.691 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T07:50:00.798 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T07:50:00.799 INFO:teuthology.task.ansible.out: 2021-02-20T07:50:00.925 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T07:50:00.969 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "zyan", "name": "zyan", "remove": false, "state": "absent"} 2021-02-20T07:50:01.054 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T07:50:01.107 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-02-20T07:50:01.181 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T07:50:01.255 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-02-20T07:50:01.298 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T07:50:01.373 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-02-20T07:50:01.427 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T07:50:01.522 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-02-20T07:50:01.544 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T07:50:01.614 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:01 +0000 (0:00:06.428) 0:00:57.110 ***** 2021-02-20T07:50:01.654 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-02-20T07:50:01.805 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-02-20T07:50:01.955 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-02-20T07:50:01.977 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T07:50:01.980 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807401.6416948-93241093424404/source", "state": "file", "uid": 0} 2021-02-20T07:50:02.017 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.403) 0:00:57.513 ***** 2021-02-20T07:50:02.057 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T07:50:02.059 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => { "msg": "Host smithi106.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-02-20T07:50:02.113 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.095) 0:00:57.609 ***** 2021-02-20T07:50:02.129 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi114.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-02-20T07:50:02.152 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T07:50:02.195 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.082) 0:00:57.691 ***** 2021-02-20T07:50:02.223 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.261 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.066) 0:00:57.757 ***** 2021-02-20T07:50:02.269 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T07:50:02.270 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-02-20T07:50:02.283 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-02-20T07:50:02.284 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.321 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.059) 0:00:57.817 ***** 2021-02-20T07:50:02.356 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-20T07:50:02.359 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.396 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.074) 0:00:57.892 ***** 2021-02-20T07:50:02.403 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-02-20T07:50:02.405 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-02-20T07:50:02.461 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.065) 0:00:57.957 ***** 2021-02-20T07:50:02.469 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-20T07:50:02.471 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.047) 0:00:58.005 ***** 2021-02-20T07:50:02.537 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-02-20T07:50:02.539 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.590 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.080) 0:00:58.086 ***** 2021-02-20T07:50:02.618 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-20T07:50:02.620 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.658 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.068) 0:00:58.154 ***** 2021-02-20T07:50:02.688 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-20T07:50:02.690 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.735 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.076) 0:00:58.231 ***** 2021-02-20T07:50:02.774 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:02.818 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.082) 0:00:58.314 ***** 2021-02-20T07:50:02.835 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-02-20T07:50:02.837 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "checksum": "5ebc9fcf8859a9c716ab10089a7bec7e44d29368", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "ee06ed012f51c2e3764171d4b9180349", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807402.4891927-77348091373884/source", "state": "file", "uid": 0} 2021-02-20T07:50:02.883 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.065) 0:00:58.379 ***** 2021-02-20T07:50:02.893 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi106.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:02.895 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:02.938 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.055) 0:00:58.434 ***** 2021-02-20T07:50:02.945 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-02-20T07:50:02.948 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => { "msg": "Host smithi114.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-02-20T07:50:02.986 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:02 +0000 (0:00:00.048) 0:00:58.482 ***** 2021-02-20T07:50:02.993 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-02-20T07:50:02.995 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.035 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.048) 0:00:58.531 ***** 2021-02-20T07:50:03.041 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-02-20T07:50:03.043 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-02-20T07:50:03.079 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.044) 0:00:58.575 ***** 2021-02-20T07:50:03.089 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.131 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.052) 0:00:58.627 ***** 2021-02-20T07:50:03.140 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-02-20T07:50:03.142 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.182 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.050) 0:00:58.678 ***** 2021-02-20T07:50:03.190 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T07:50:03.192 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:03.229 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.047) 0:00:58.725 ***** 2021-02-20T07:50:03.237 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-02-20T07:50:03.239 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.277 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.047) 0:00:58.773 ***** 2021-02-20T07:50:03.285 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-20T07:50:03.287 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.322 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.045) 0:00:58.818 ***** 2021-02-20T07:50:03.329 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-02-20T07:50:03.331 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.368 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.045) 0:00:58.864 ***** 2021-02-20T07:50:03.375 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:03.416 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.047) 0:00:58.912 ***** 2021-02-20T07:50:03.424 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-02-20T07:50:03.426 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.463 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.046) 0:00:58.959 ***** 2021-02-20T07:50:03.470 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-20T07:50:03.472 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.046) 0:00:59.005 ***** 2021-02-20T07:50:03.517 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.554 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.044) 0:00:59.050 ***** 2021-02-20T07:50:03.568 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-02-20T07:50:03.569 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.604 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.049) 0:00:59.100 ***** 2021-02-20T07:50:03.612 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-02-20T07:50:03.614 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.650 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.046) 0:00:59.146 ***** 2021-02-20T07:50:03.656 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-20T07:50:03.658 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.694 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.043) 0:00:59.190 ***** 2021-02-20T07:50:03.700 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-02-20T07:50:03.702 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.749 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.055) 0:00:59.245 ***** 2021-02-20T07:50:03.758 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-20T07:50:03.760 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.800 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.050) 0:00:59.296 ***** 2021-02-20T07:50:03.807 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-02-20T07:50:03.809 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:03.845 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.045) 0:00:59.341 ***** 2021-02-20T07:50:03.852 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-20T07:50:03.854 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:03.910 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.064) 0:00:59.406 ***** 2021-02-20T07:50:03.918 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-02-20T07:50:03.920 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:03.957 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:03 +0000 (0:00:00.046) 0:00:59.453 ***** 2021-02-20T07:50:03.964 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T07:50:03.965 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.004 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.047) 0:00:59.500 ***** 2021-02-20T07:50:04.012 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-02-20T07:50:04.014 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.066 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.061) 0:00:59.562 ***** 2021-02-20T07:50:04.074 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-20T07:50:04.075 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.130 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.064) 0:00:59.626 ***** 2021-02-20T07:50:04.139 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-02-20T07:50:04.141 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.190 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.059) 0:00:59.686 ***** 2021-02-20T07:50:04.204 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.241 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.051) 0:00:59.737 ***** 2021-02-20T07:50:04.251 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-02-20T07:50:04.253 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:04.290 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.049) 0:00:59.786 ***** 2021-02-20T07:50:04.297 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-02-20T07:50:04.299 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.353 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.062) 0:00:59.849 ***** 2021-02-20T07:50:04.361 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-02-20T07:50:04.363 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.405 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.052) 0:00:59.901 ***** 2021-02-20T07:50:04.417 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-02-20T07:50:04.419 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.477 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.072) 0:00:59.973 ***** 2021-02-20T07:50:04.484 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-02-20T07:50:04.487 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:04.526 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.049) 0:01:00.022 ***** 2021-02-20T07:50:04.535 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T07:50:04.537 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.581 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.054) 0:01:00.077 ***** 2021-02-20T07:50:04.588 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-02-20T07:50:04.590 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.637 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.055) 0:01:00.133 ***** 2021-02-20T07:50:04.644 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T07:50:04.648 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:04.687 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.049) 0:01:00.183 ***** 2021-02-20T07:50:04.694 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.731 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.044) 0:01:00.227 ***** 2021-02-20T07:50:04.739 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.780 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.047) 0:01:00.275 ***** 2021-02-20T07:50:04.788 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-02-20T07:50:04.790 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.827 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.047) 0:01:00.323 ***** 2021-02-20T07:50:04.834 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T07:50:04.837 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.874 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.047) 0:01:00.370 ***** 2021-02-20T07:50:04.881 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-02-20T07:50:04.882 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.921 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.046) 0:01:00.417 ***** 2021-02-20T07:50:04.927 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-20T07:50:04.929 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:04.968 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:04 +0000 (0:00:00.047) 0:01:00.464 ***** 2021-02-20T07:50:04.976 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-02-20T07:50:04.978 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:05.015 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.047) 0:01:00.511 ***** 2021-02-20T07:50:05.022 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T07:50:05.024 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.077 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.061) 0:01:00.573 ***** 2021-02-20T07:50:05.092 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T07:50:05.095 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.129 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.051) 0:01:00.625 ***** 2021-02-20T07:50:05.135 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-20T07:50:05.137 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.184 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.055) 0:01:00.680 ***** 2021-02-20T07:50:05.193 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-02-20T07:50:05.195 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.237 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.053) 0:01:00.733 ***** 2021-02-20T07:50:05.243 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T07:50:05.245 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.291 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.053) 0:01:00.787 ***** 2021-02-20T07:50:05.297 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-02-20T07:50:05.299 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.336 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.045) 0:01:00.832 ***** 2021-02-20T07:50:05.363 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-02-20T07:50:05.365 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.404 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.068) 0:01:00.900 ***** 2021-02-20T07:50:05.444 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T07:50:05.446 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.012638", "end": "2021-02-20 07:50:05.445919", "rc": 0, "start": "2021-02-20 07:50:05.433281", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.480 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.075) 0:01:00.976 ***** 2021-02-20T07:50:05.515 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.034) 0:01:01.011 ***** 2021-02-20T07:50:05.521 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-20T07:50:05.523 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.567 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.052) 0:01:01.063 ***** 2021-02-20T07:50:05.574 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-02-20T07:50:05.575 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.613 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.046) 0:01:01.109 ***** 2021-02-20T07:50:05.619 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-02-20T07:50:05.621 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.658 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.044) 0:01:01.154 ***** 2021-02-20T07:50:05.665 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-02-20T07:50:05.666 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:05.723 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.065) 0:01:01.219 ***** 2021-02-20T07:50:05.745 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T07:50:05.747 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.748 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.751 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.788 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.064) 0:01:01.284 ***** 2021-02-20T07:50:05.795 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-02-20T07:50:05.797 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.834 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.045) 0:01:01.330 ***** 2021-02-20T07:50:05.840 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T07:50:05.842 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.884 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.049) 0:01:01.380 ***** 2021-02-20T07:50:05.893 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-02-20T07:50:05.894 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.942 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.058) 0:01:01.438 ***** 2021-02-20T07:50:05.952 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-20T07:50:05.954 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:05.995 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:05 +0000 (0:00:00.052) 0:01:01.491 ***** 2021-02-20T07:50:06.003 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-02-20T07:50:06.004 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.040 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.045) 0:01:01.536 ***** 2021-02-20T07:50:06.048 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-20T07:50:06.050 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.089 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.048) 0:01:01.585 ***** 2021-02-20T07:50:06.099 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-02-20T07:50:06.101 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.101 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.142 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.052) 0:01:01.638 ***** 2021-02-20T07:50:06.151 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-20T07:50:06.152 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.049) 0:01:01.688 ***** 2021-02-20T07:50:06.199 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-02-20T07:50:06.200 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.238 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.046) 0:01:01.734 ***** 2021-02-20T07:50:06.244 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-20T07:50:06.246 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.284 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.046) 0:01:01.780 ***** 2021-02-20T07:50:06.291 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-02-20T07:50:06.293 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.336 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.051) 0:01:01.832 ***** 2021-02-20T07:50:06.495 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T07:50:06.497 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.014178", "end": "2021-02-20 07:50:06.461656", "rc": 0, "start": "2021-02-20 07:50:06.447478", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-02-20T07:50:06.557 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.221) 0:01:02.053 ***** 2021-02-20T07:50:06.567 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.080125", "end": "2021-02-20 07:50:06.500447", "rc": 0, "start": "2021-02-20 07:50:06.420322", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:06.630 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.073) 0:01:02.126 ***** 2021-02-20T07:50:06.638 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-02-20T07:50:06.640 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.678 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.047) 0:01:02.174 ***** 2021-02-20T07:50:06.689 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-20T07:50:06.691 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.731 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.053) 0:01:02.227 ***** 2021-02-20T07:50:06.740 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.779 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.048) 0:01:02.275 ***** 2021-02-20T07:50:06.821 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-02-20T07:50:06.823 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.823 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.834 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.835 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.872 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.092) 0:01:02.368 ***** 2021-02-20T07:50:06.900 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-02-20T07:50:06.902 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:06.936 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.064) 0:01:02.432 ***** 2021-02-20T07:50:06.943 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T07:50:06.945 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.039093", "end": "2021-02-20 07:50:06.909414", "rc": 0, "start": "2021-02-20 07:50:06.870321", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:06.982 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:06 +0000 (0:00:00.045) 0:01:02.478 ***** 2021-02-20T07:50:06.990 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-02-20T07:50:06.992 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:07.028 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.045) 0:01:02.524 ***** 2021-02-20T07:50:07.056 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-02-20T07:50:07.058 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:07.106 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.077) 0:01:02.602 ***** 2021-02-20T07:50:07.136 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-02-20T07:50:07.138 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:07.184 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.078) 0:01:02.680 ***** 2021-02-20T07:50:07.212 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-02-20T07:50:07.214 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:07.248 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.063) 0:01:02.744 ***** 2021-02-20T07:50:07.530 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.152643", "end": "2021-02-20 07:50:07.500403", "rc": 0, "start": "2021-02-20 07:50:07.347760", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:07.567 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.319) 0:01:03.064 ***** 2021-02-20T07:50:07.595 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-02-20T07:50:07.597 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:07.633 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.064) 0:01:03.129 ***** 2021-02-20T07:50:07.812 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-02-20T07:50:07.814 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.044326", "end": "2021-02-20 07:50:07.787491", "rc": 0, "start": "2021-02-20 07:50:07.743165", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:07.850 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:07 +0000 (0:00:00.217) 0:01:03.346 ***** 2021-02-20T07:50:10.714 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-02-20T07:50:10.716 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:10.757 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:10 +0000 (0:00:02.907) 0:01:06.253 ***** 2021-02-20T07:50:11.391 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:11.437 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:11 +0000 (0:00:00.680) 0:01:06.933 ***** 2021-02-20T07:50:11.615 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-20T07:50:11.617 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:11.653 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:11 +0000 (0:00:00.215) 0:01:07.149 ***** 2021-02-20T07:50:11.682 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-20T07:50:11.684 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:11.724 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:11 +0000 (0:00:00.070) 0:01:07.220 ***** 2021-02-20T07:50:12.311 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-02-20T07:50:12.313 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:12.354 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:12 +0000 (0:00:00.630) 0:01:07.850 ***** 2021-02-20T07:50:12.364 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-20T07:50:12.365 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T07:50:12.405 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:12 +0000 (0:00:00.051) 0:01:07.901 ***** 2021-02-20T07:50:12.415 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-02-20T07:50:12.417 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:12.459 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:12 +0000 (0:00:00.053) 0:01:07.955 ***** 2021-02-20T07:50:13.103 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-02-20T07:50:13.105 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-02-20T07:50:13.151 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:13 +0000 (0:00:00.691) 0:01:08.647 ***** 2021-02-20T07:50:13.313 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-20T07:50:13.314 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:13.357 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:13 +0000 (0:00:00.205) 0:01:08.853 ***** 2021-02-20T07:50:13.712 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T07:50:13.713 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T07:50:13.997 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T07:50:14.041 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-02-20T07:50:14.043 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:14.078 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:14 +0000 (0:00:00.721) 0:01:09.574 ***** 2021-02-20T07:50:14.263 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T07:50:14.265 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T07:50:14.308 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:14 +0000 (0:00:00.230) 0:01:09.804 ***** 2021-02-20T07:50:14.431 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-02-20T07:50:14.443 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-20T07:50:14.445 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:50:14.486 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:14 +0000 (0:00:00.177) 0:01:09.981 ***** 2021-02-20T07:50:14.770 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T07:50:14.772 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-02-20T07:50:14.835 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T07:50:14.837 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T07:50:15.038 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-02-20T07:50:15.040 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-02-20T07:50:15.082 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.596) 0:01:10.578 ***** 2021-02-20T07:50:15.132 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T07:50:15.132 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T07:50:15.218 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-02-20T07:50:15.220 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:50:15.259 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.176) 0:01:10.755 ***** 2021-02-20T07:50:15.409 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T07:50:15.411 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T07:50:15.450 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.191) 0:01:10.946 ***** 2021-02-20T07:50:15.484 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-20T07:50:15.486 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T07:50:15.525 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.075) 0:01:11.021 ***** 2021-02-20T07:50:15.556 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-20T07:50:15.558 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.598 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.073) 0:01:11.095 ***** 2021-02-20T07:50:15.606 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T07:50:15.608 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-02-20T07:50:15.629 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:15.670 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.071) 0:01:11.166 ***** 2021-02-20T07:50:15.709 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-20T07:50:15.712 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.721 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.732 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.734 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.771 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.101) 0:01:11.267 ***** 2021-02-20T07:50:15.798 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T07:50:15.800 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:15.853 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:15 +0000 (0:00:00.081) 0:01:11.349 ***** 2021-02-20T07:50:15.860 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-02-20T07:50:15.861 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-02-20T07:50:16.116 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-02-20T07:50:16.162 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.308) 0:01:11.658 ***** 2021-02-20T07:50:16.202 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-02-20T07:50:16.204 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-02-20T07:50:16.239 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.077) 0:01:11.735 ***** 2021-02-20T07:50:16.278 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-02-20T07:50:16.281 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.319 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.079) 0:01:11.815 ***** 2021-02-20T07:50:16.358 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-02-20T07:50:16.360 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:16.399 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.080) 0:01:11.895 ***** 2021-02-20T07:50:16.443 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-02-20T07:50:16.444 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.454 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.457 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.466 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.506 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.106) 0:01:12.002 ***** 2021-02-20T07:50:16.543 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-02-20T07:50:16.545 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:16.580 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.074) 0:01:12.076 ***** 2021-02-20T07:50:16.811 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-02-20T07:50:16.813 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:16.849 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:16 +0000 (0:00:00.268) 0:01:12.345 ***** 2021-02-20T07:50:17.184 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-20T07:50:17.186 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T07:50:17.256 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:17 +0000 (0:00:00.403) 0:01:12.749 ***** 2021-02-20T07:50:17.603 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-02-20T07:50:17.605 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T07:50:17.644 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:17 +0000 (0:00:00.390) 0:01:13.140 ***** 2021-02-20T07:50:17.679 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:17 +0000 (0:00:00.034) 0:01:13.175 ***** 2021-02-20T07:50:17.792 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T07:50:17.812 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T07:50:17.908 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T07:50:17.948 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:17 +0000 (0:00:00.268) 0:01:13.444 ***** 2021-02-20T07:50:17.986 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-02-20T07:50:17.988 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:18.032 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.084) 0:01:13.528 ***** 2021-02-20T07:50:18.039 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-02-20T07:50:18.040 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-02-20T07:50:18.079 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.046) 0:01:13.575 ***** 2021-02-20T07:50:18.085 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:18.123 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.043) 0:01:13.619 ***** 2021-02-20T07:50:18.161 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T07:50:18.163 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:18.206 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.082) 0:01:13.702 ***** 2021-02-20T07:50:18.409 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-02-20T07:50:18.412 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-02-20T07:50:18.449 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.243) 0:01:13.946 ***** 2021-02-20T07:50:18.584 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-02-20T07:50:18.586 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T07:50:18.645 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.195) 0:01:14.141 ***** 2021-02-20T07:50:18.657 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-02-20T07:50:18.658 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-02-20T07:50:18.723 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-02-20T07:50:18.775 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.130) 0:01:14.271 ***** 2021-02-20T07:50:18.818 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:18.856 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.081) 0:01:14.352 ***** 2021-02-20T07:50:18.911 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi114.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-02-20T07:50:18.953 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:18 +0000 (0:00:00.096) 0:01:14.449 ***** 2021-02-20T07:50:18.993 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-02-20T07:50:18.995 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.047 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.094) 0:01:14.543 ***** 2021-02-20T07:50:19.318 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-20T07:50:19.320 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:48:49 UTC", "ActiveEnterTimestampMonotonic": "22139637", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target network.target sysinit.target systemd-journald.socket -.mount time-sync.target system.slice local-fs.target basic.target remote-fs.target var-run.mount systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:49 UTC", "AssertTimestampMonotonic": "22137020", "Before": "plymouth-quit.service getty@tty1.service xdm.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:49 UTC", "ConditionTimestampMonotonic": "22137020", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-02-20T07:50:19.322 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1183", "ExecMainStartTimestamp": "Sat 2021-02-20 07:48:49 UTC", "ExecMainStartTimestampMonotonic": "22139596", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 07:48:49 UTC] ; stop_time=[n/a] ; pid=1183 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:49 UTC", "InactiveExitTimestampMonotonic": "22139637", "InvocationID": "158d1da071d948f98812b17b86a860e0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-02-20T07:50:19.326 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1183", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-02-20T07:50:19.328 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:48:49 UTC", "StateChangeTimestampMonotonic": "22139637", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:48:49 UTC", "WatchdogTimestampMonotonic": "22139635", "WatchdogUSec": "0"}} 2021-02-20T07:50:19.407 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.358) 0:01:14.902 ***** 2021-02-20T07:50:19.471 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-02-20T07:50:19.476 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.069) 0:01:14.971 ***** 2021-02-20T07:50:19.482 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-20T07:50:19.484 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.546 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.070) 0:01:15.042 ***** 2021-02-20T07:50:19.591 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-20T07:50:19.593 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.632 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.086) 0:01:15.128 ***** 2021-02-20T07:50:19.678 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-20T07:50:19.680 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.762 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.130) 0:01:15.258 ***** 2021-02-20T07:50:19.793 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-02-20T07:50:19.795 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:48:55 UTC", "ActiveEnterTimestampMonotonic": "21749807", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service network.target time-sync.target sysinit.target remote-fs.target nss-lookup.target systemd-journald.socket var-run.mount basic.target system.slice local-fs.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:55 UTC", "AssertTimestampMonotonic": "21748824", "Before": "getty@tty1.service multi-user.target plymouth-quit.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:55 UTC", "ConditionTimestampMonotonic": "21748824", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-02-20T07:50:19.799 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1163", "ExecMainStartTimestamp": "Sat 2021-02-20 07:48:55 UTC", "ExecMainStartTimestampMonotonic": "21749759", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sat 2021-02-20 07:48:55 UTC] ; stop_time=[n/a] ; pid=1163 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:55 UTC", "InactiveExitTimestampMonotonic": "21749807", "InvocationID": "9778b35b669a4482828a4135220e9f7c", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2021-02-20T07:50:19.802 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1163", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2021-02-20T07:50:19.804 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:48:55 UTC", "StateChangeTimestampMonotonic": "21749807", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:48:55 UTC", "WatchdogTimestampMonotonic": "21749806", "WatchdogUSec": "0"}} 2021-02-20T07:50:19.833 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.071) 0:01:15.329 ***** 2021-02-20T07:50:19.840 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.886 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.052) 0:01:15.382 ***** 2021-02-20T07:50:19.920 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-02-20T07:50:19.922 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:19.978 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:19 +0000 (0:00:00.091) 0:01:15.474 ***** 2021-02-20T07:50:19.987 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-02-20T07:50:19.988 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.043 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.065) 0:01:15.539 ***** 2021-02-20T07:50:20.071 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-02-20T07:50:20.077 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.123 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.080) 0:01:15.619 ***** 2021-02-20T07:50:20.296 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-20T07:50:20.298 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.335 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.211) 0:01:15.831 ***** 2021-02-20T07:50:20.343 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-02-20T07:50:20.345 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.391 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.055) 0:01:15.887 ***** 2021-02-20T07:50:20.400 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.450 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.059) 0:01:15.946 ***** 2021-02-20T07:50:20.459 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.505 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.054) 0:01:16.001 ***** 2021-02-20T07:50:20.513 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-20T07:50:20.514 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.551 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.046) 0:01:16.047 ***** 2021-02-20T07:50:20.558 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.597 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.045) 0:01:16.093 ***** 2021-02-20T07:50:20.604 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-02-20T07:50:20.606 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.649 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.051) 0:01:16.145 ***** 2021-02-20T07:50:20.656 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-02-20T07:50:20.657 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.703 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.054) 0:01:16.199 ***** 2021-02-20T07:50:20.711 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-20T07:50:20.713 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.752 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.049) 0:01:16.248 ***** 2021-02-20T07:50:20.760 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-02-20T07:50:20.762 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.799 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.046) 0:01:16.295 ***** 2021-02-20T07:50:20.806 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-20T07:50:20.808 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T07:50:20.844 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.045) 0:01:16.341 ***** 2021-02-20T07:50:20.853 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-02-20T07:50:20.855 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.893 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.048) 0:01:16.389 ***** 2021-02-20T07:50:20.899 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-20T07:50:20.901 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T07:50:20.938 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.045) 0:01:16.434 ***** 2021-02-20T07:50:20.953 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-02-20T07:50:20.955 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:20.990 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:20 +0000 (0:00:00.052) 0:01:16.486 ***** 2021-02-20T07:50:21.008 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-20T07:50:21.009 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T07:50:21.051 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.061) 0:01:16.547 ***** 2021-02-20T07:50:21.058 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-02-20T07:50:21.060 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:21.098 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.046) 0:01:16.594 ***** 2021-02-20T07:50:21.107 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-20T07:50:21.109 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T07:50:21.160 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.061) 0:01:16.656 ***** 2021-02-20T07:50:21.169 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-02-20T07:50:21.170 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-02-20T07:50:21.212 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.052) 0:01:16.708 ***** 2021-02-20T07:50:21.262 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-02-20T07:50:21.265 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-02-20T07:50:21.310 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.098) 0:01:16.806 ***** 2021-02-20T07:50:21.320 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-20T07:50:21.323 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:50:21.361 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.050) 0:01:16.857 ***** 2021-02-20T07:50:21.371 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-02-20T07:50:21.373 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-02-20T07:50:21.437 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.075) 0:01:16.933 ***** 2021-02-20T07:50:21.495 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-02-20T07:50:21.498 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-02-20T07:50:21.537 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.100) 0:01:17.033 ***** 2021-02-20T07:50:21.556 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T07:50:21.558 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T07:50:21.623 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.085) 0:01:17.119 ***** 2021-02-20T07:50:21.651 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T07:50:21.654 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:21.701 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.077) 0:01:17.197 ***** 2021-02-20T07:50:21.720 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-02-20T07:50:21.722 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-02-20T07:50:21.780 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.079) 0:01:17.276 ***** 2021-02-20T07:50:21.788 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T07:50:21.790 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:21.843 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.062) 0:01:17.339 ***** 2021-02-20T07:50:21.924 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-02-20T07:50:21.926 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-02-20T07:50:21.966 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:21 +0000 (0:00:00.123) 0:01:17.462 ***** 2021-02-20T07:50:21.994 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-02-20T07:50:21.996 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:22.036 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.069) 0:01:17.532 ***** 2021-02-20T07:50:22.043 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-20T07:50:22.044 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T07:50:22.082 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.045) 0:01:17.578 ***** 2021-02-20T07:50:22.088 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-02-20T07:50:22.090 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:22.128 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.046) 0:01:17.624 ***** 2021-02-20T07:50:22.315 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-02-20T07:50:22.317 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-02-20T07:50:22.351 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.223) 0:01:17.847 ***** 2021-02-20T07:50:22.507 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-20T07:50:22.508 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T07:50:22.546 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.194) 0:01:18.042 ***** 2021-02-20T07:50:22.614 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-20T07:50:22.616 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi106"}, "changed": false} 2021-02-20T07:50:22.676 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.127) 0:01:18.170 ***** 2021-02-20T07:50:22.794 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-02-20T07:50:22.797 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-02-20T07:50:22.884 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.210) 0:01:18.380 ***** 2021-02-20T07:50:22.923 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-02-20T07:50:22.925 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi114"}, "changed": false} 2021-02-20T07:50:22.960 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:22 +0000 (0:00:00.075) 0:01:18.456 ***** 2021-02-20T07:50:23.753 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-02-20T07:50:23.755 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi114.front.sepia.ceph.com", "ansible_hostname": "smithi114", "ansible_nodename": "smithi114"}, "changed": true, "name": "smithi114"} 2021-02-20T07:50:23.789 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:23 +0000 (0:00:00.829) 0:01:19.285 ***** 2021-02-20T07:50:23.892 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi106.front.sepia.ceph.com", "ansible_hostname": "smithi106", "ansible_nodename": "smithi106"}, "changed": true, "name": "smithi106"} 2021-02-20T07:50:23.931 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:23 +0000 (0:00:00.141) 0:01:19.427 ***** 2021-02-20T07:50:24.119 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-02-20T07:50:24.121 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T07:50:24.156 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:24 +0000 (0:00:00.225) 0:01:19.652 ***** 2021-02-20T07:50:24.258 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-02-20T07:50:24.295 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:24 +0000 (0:00:00.138) 0:01:19.791 ***** 2021-02-20T07:50:24.481 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-02-20T07:50:24.483 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T07:50:24.522 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:24 +0000 (0:00:00.226) 0:01:20.018 ***** 2021-02-20T07:50:24.639 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-02-20T07:50:24.685 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:24 +0000 (0:00:00.162) 0:01:20.181 ***** 2021-02-20T07:50:25.076 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-02-20T07:50:25.077 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T07:50:25.120 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.435) 0:01:20.616 ***** 2021-02-20T07:50:25.149 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-20T07:50:25.150 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.192 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.072) 0:01:20.688 ***** 2021-02-20T07:50:25.219 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T07:50:25.221 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-02-20T07:50:25.262 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.069) 0:01:20.758 ***** 2021-02-20T07:50:25.314 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.051) 0:01:20.810 ***** 2021-02-20T07:50:25.323 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-02-20T07:50:25.325 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.363 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.048) 0:01:20.859 ***** 2021-02-20T07:50:25.369 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T07:50:25.371 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.416 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.053) 0:01:20.912 ***** 2021-02-20T07:50:25.424 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T07:50:25.426 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.464 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.048) 0:01:20.960 ***** 2021-02-20T07:50:25.471 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T07:50:25.473 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.044) 0:01:21.005 ***** 2021-02-20T07:50:25.516 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T07:50:25.517 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.561 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.052) 0:01:21.057 ***** 2021-02-20T07:50:25.568 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T07:50:25.570 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.621 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.059) 0:01:21.117 ***** 2021-02-20T07:50:25.630 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T07:50:25.632 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.686 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.064) 0:01:21.182 ***** 2021-02-20T07:50:25.693 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T07:50:25.695 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.737 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.051) 0:01:21.233 ***** 2021-02-20T07:50:25.744 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T07:50:25.746 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.793 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.056) 0:01:21.289 ***** 2021-02-20T07:50:25.802 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T07:50:25.804 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.840 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.047) 0:01:21.336 ***** 2021-02-20T07:50:25.847 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T07:50:25.849 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.886 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.045) 0:01:21.382 ***** 2021-02-20T07:50:25.893 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-20T07:50:25.895 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:25.951 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:25 +0000 (0:00:00.065) 0:01:21.447 ***** 2021-02-20T07:50:25.964 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T07:50:25.966 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.020 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.068) 0:01:21.516 ***** 2021-02-20T07:50:26.031 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-20T07:50:26.079 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.058) 0:01:21.575 ***** 2021-02-20T07:50:26.087 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-02-20T07:50:26.089 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.137 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.058) 0:01:21.633 ***** 2021-02-20T07:50:26.145 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-20T07:50:26.184 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.046) 0:01:21.680 ***** 2021-02-20T07:50:26.193 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-02-20T07:50:26.230 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.045) 0:01:21.726 ***** 2021-02-20T07:50:26.237 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-20T07:50:26.275 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.045) 0:01:21.771 ***** 2021-02-20T07:50:26.282 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-02-20T07:50:26.321 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.046) 0:01:21.817 ***** 2021-02-20T07:50:26.329 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.369 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.048) 0:01:21.866 ***** 2021-02-20T07:50:26.377 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-02-20T07:50:26.425 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.055) 0:01:21.921 ***** 2021-02-20T07:50:26.432 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-20T07:50:26.434 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.471 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.046) 0:01:21.967 ***** 2021-02-20T07:50:26.478 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-02-20T07:50:26.480 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.523 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.051) 0:01:22.019 ***** 2021-02-20T07:50:26.531 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-20T07:50:26.532 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.575 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.051) 0:01:22.071 ***** 2021-02-20T07:50:26.581 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-02-20T07:50:26.584 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.621 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.046) 0:01:22.117 ***** 2021-02-20T07:50:26.628 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:50:26.630 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.673 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.052) 0:01:22.169 ***** 2021-02-20T07:50:26.683 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-02-20T07:50:26.685 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.723 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.049) 0:01:22.219 ***** 2021-02-20T07:50:26.731 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-20T07:50:26.733 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.771 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.048) 0:01:22.267 ***** 2021-02-20T07:50:26.777 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:50:26.779 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.822 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.050) 0:01:22.318 ***** 2021-02-20T07:50:26.829 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-20T07:50:26.831 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.882 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.059) 0:01:22.378 ***** 2021-02-20T07:50:26.899 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-02-20T07:50:26.900 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.944 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.062) 0:01:22.440 ***** 2021-02-20T07:50:26.954 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:26.992 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:26 +0000 (0:00:00.047) 0:01:22.488 ***** 2021-02-20T07:50:27.000 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-02-20T07:50:27.001 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.041 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.048) 0:01:22.537 ***** 2021-02-20T07:50:27.051 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-20T07:50:27.053 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.092 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.051) 0:01:22.588 ***** 2021-02-20T07:50:27.100 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-02-20T07:50:27.101 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.140 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.048) 0:01:22.636 ***** 2021-02-20T07:50:27.147 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T07:50:27.149 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.187 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.046) 0:01:22.683 ***** 2021-02-20T07:50:27.195 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-02-20T07:50:27.196 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.231 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.044) 0:01:22.727 ***** 2021-02-20T07:50:27.238 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T07:50:27.239 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.278 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.047) 0:01:22.774 ***** 2021-02-20T07:50:27.284 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T07:50:27.286 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.323 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.044) 0:01:22.819 ***** 2021-02-20T07:50:27.332 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.371 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.048) 0:01:22.867 ***** 2021-02-20T07:50:27.377 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T07:50:27.379 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.428 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.057) 0:01:22.924 ***** 2021-02-20T07:50:27.435 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-20T07:50:27.436 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.485 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.056) 0:01:22.981 ***** 2021-02-20T07:50:27.493 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T07:50:27.495 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.540 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.055) 0:01:23.036 ***** 2021-02-20T07:50:27.557 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T07:50:27.560 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.615 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.075) 0:01:23.111 ***** 2021-02-20T07:50:27.623 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-02-20T07:50:27.626 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.687 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.071) 0:01:23.183 ***** 2021-02-20T07:50:27.698 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.741 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.054) 0:01:23.237 ***** 2021-02-20T07:50:27.748 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T07:50:27.750 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.798 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.056) 0:01:23.294 ***** 2021-02-20T07:50:27.819 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-20T07:50:27.822 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.871 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.073) 0:01:23.367 ***** 2021-02-20T07:50:27.881 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T07:50:27.884 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.927 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.055) 0:01:23.423 ***** 2021-02-20T07:50:27.939 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-20T07:50:27.941 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:27.985 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:27 +0000 (0:00:00.058) 0:01:23.481 ***** 2021-02-20T07:50:27.994 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-02-20T07:50:27.996 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.046 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.060) 0:01:23.542 ***** 2021-02-20T07:50:28.056 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-20T07:50:28.058 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.094 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.047) 0:01:23.590 ***** 2021-02-20T07:50:28.101 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-02-20T07:50:28.103 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.140 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.046) 0:01:23.636 ***** 2021-02-20T07:50:28.147 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-20T07:50:28.149 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.201 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.060) 0:01:23.697 ***** 2021-02-20T07:50:28.207 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-02-20T07:50:28.209 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.054) 0:01:23.751 ***** 2021-02-20T07:50:28.264 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-20T07:50:28.266 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.301 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.046) 0:01:23.797 ***** 2021-02-20T07:50:28.308 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-02-20T07:50:28.311 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.357 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.055) 0:01:23.853 ***** 2021-02-20T07:50:28.366 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-20T07:50:28.368 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.411 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.053) 0:01:23.907 ***** 2021-02-20T07:50:28.417 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-02-20T07:50:28.419 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.457 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.046) 0:01:23.953 ***** 2021-02-20T07:50:28.466 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-20T07:50:28.467 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.502 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.045) 0:01:23.999 ***** 2021-02-20T07:50:28.509 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-02-20T07:50:28.510 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.548 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.045) 0:01:24.044 ***** 2021-02-20T07:50:28.555 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-20T07:50:28.557 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.596 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.047) 0:01:24.092 ***** 2021-02-20T07:50:28.603 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-02-20T07:50:28.604 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.645 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.049) 0:01:24.141 ***** 2021-02-20T07:50:28.683 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-02-20T07:50:28.685 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:28.725 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.079) 0:01:24.221 ***** 2021-02-20T07:50:28.753 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-02-20T07:50:28.755 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005146", "end": "2021-02-20 07:50:28.717019", "rc": 0, "start": "2021-02-20 07:50:28.711873", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:28.791 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.065) 0:01:24.286 ***** 2021-02-20T07:50:28.818 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-20T07:50:28.865 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.074) 0:01:24.361 ***** 2021-02-20T07:50:28.884 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-02-20T07:50:28.886 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005160", "end": "2021-02-20 07:50:28.880175", "rc": 0, "start": "2021-02-20 07:50:28.875015", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:28.923 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:28 +0000 (0:00:00.057) 0:01:24.419 ***** 2021-02-20T07:50:28.952 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-02-20T07:50:29.008 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:29 +0000 (0:00:00.085) 0:01:24.504 ***** 2021-02-20T07:50:29.213 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-02-20T07:50:29.215 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T07:50:29.267 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:29 +0000 (0:00:00.259) 0:01:24.763 ***** 2021-02-20T07:50:29.311 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:29.365 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:29 +0000 (0:00:00.098) 0:01:24.862 ***** 2021-02-20T07:50:30.378 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-02-20T07:50:30.380 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-02-20T07:50:30.438 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:30 +0000 (0:00:01.072) 0:01:25.934 ***** 2021-02-20T07:50:30.476 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-02-20T07:50:30.478 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:30.513 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:30 +0000 (0:00:00.075) 0:01:26.009 ***** 2021-02-20T07:50:30.754 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-02-20T07:50:30.755 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-02-20T07:50:31.526 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-02-20T07:50:31.793 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T07:50:31.833 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:31 +0000 (0:00:01.319) 0:01:27.329 ***** 2021-02-20T07:50:32.504 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-02-20T07:50:32.542 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:32 +0000 (0:00:00.708) 0:01:28.038 ***** 2021-02-20T07:50:32.758 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-20T07:50:32.760 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:32.796 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:32 +0000 (0:00:00.254) 0:01:28.292 ***** 2021-02-20T07:50:32.824 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-20T07:50:32.866 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:32 +0000 (0:00:00.069) 0:01:28.362 ***** 2021-02-20T07:50:33.045 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-20T07:50:33.047 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.026793", "end": "2021-02-20 07:50:33.010220", "rc": 0, "start": "2021-02-20 07:50:32.983427", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:33.081 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:33 +0000 (0:00:00.214) 0:01:28.577 ***** 2021-02-20T07:50:33.564 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-02-20T07:50:33.566 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:33.605 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:33 +0000 (0:00:00.524) 0:01:29.101 ***** 2021-02-20T07:50:33.633 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-02-20T07:50:33.679 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:33 +0000 (0:00:00.074) 0:01:29.175 ***** 2021-02-20T07:50:33.856 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-02-20T07:50:33.857 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035227", "end": "2021-02-20 07:50:33.843605", "rc": 0, "start": "2021-02-20 07:50:33.808378", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:50:33.901 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:33 +0000 (0:00:00.221) 0:01:29.397 ***** 2021-02-20T07:50:36.062 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-02-20T07:50:36.063 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:36.099 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:36 +0000 (0:00:02.198) 0:01:31.595 ***** 2021-02-20T07:50:36.890 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:36.937 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:36 +0000 (0:00:00.836) 0:01:32.432 ***** 2021-02-20T07:50:45.309 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-02-20T07:50:45.311 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directorie 2021-02-20T07:50:45.313 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-02-20T07:50:45.314 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T07:50:45.349 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:45 +0000 (0:00:08.413) 0:01:40.845 ***** 2021-02-20T07:50:46.260 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T07:50:46.262 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:46.303 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:46 +0000 (0:00:00.954) 0:01:41.799 ***** 2021-02-20T07:50:46.343 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-20T07:50:46.345 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading databa 2021-02-20T07:50:46.346 INFO:teuthology.task.ansible.out:se ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137610 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 34 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading da 2021-02-20T07:50:46.348 INFO:teuthology.task.ansible.out:tabase ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137610 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.34) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.34) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T07:50:46.382 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:46 +0000 (0:00:00.079) 0:01:41.878 ***** 2021-02-20T07:50:46.419 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:46 +0000 (0:00:00.036) 0:01:41.915 ***** 2021-02-20T07:50:47.317 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:47.358 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:47 +0000 (0:00:00.938) 0:01:42.854 ***** 2021-02-20T07:50:47.388 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-02-20T07:50:47.390 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:47.447 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:47 +0000 (0:00:00.089) 0:01:42.943 ***** 2021-02-20T07:50:55.824 INFO:teuthology.task.ansible.out: 2021-02-20T07:50:55.826 INFO:teuthology.task.ansible.out:TASK [testnode : Install packages] ********************************************* changed: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 2s (6038 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database . 2021-02-20T07:50:55.828 INFO:teuthology.task.ansible.out:.. 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing triggers 2021-02-20T07:50:55.830 INFO:teuthology.task.ansible.out:for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 2s (6038 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Readin 2021-02-20T07:50:55.831 INFO:teuthology.task.ansible.out:g database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T07:50:55.864 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:55 +0000 (0:00:08.416) 0:01:51.360 ***** 2021-02-20T07:50:56.131 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]\nFetched 10.4 MB in 1s (9689 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-02-20T07:50:56.133 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137468 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-02-20T07:50:56.135 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 34 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.35 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.35 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.35 [5194 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.35 [868 kB]", "Fetched 10.4 MB in 1s (9689 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-02-20T07:50:56.136 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137468 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.35_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.35) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.35) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-02-20T07:50:56.175 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:56 +0000 (0:00:00.311) 0:01:51.671 ***** 2021-02-20T07:50:56.747 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-20T07:50:56.751 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:50:56.788 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:56 +0000 (0:00:00.613) 0:01:52.285 ***** 2021-02-20T07:50:56.955 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T07:50:56.957 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T07:50:56.996 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:56 +0000 (0:00:00.207) 0:01:52.492 ***** 2021-02-20T07:50:57.025 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-20T07:50:57.027 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:57.062 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.065) 0:01:52.558 ***** 2021-02-20T07:50:57.089 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-02-20T07:50:57.091 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:50:57.132 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.069) 0:01:52.628 ***** 2021-02-20T07:50:57.202 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-02-20T07:50:57.204 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-02-20T07:50:57.243 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.110) 0:01:52.739 ***** 2021-02-20T07:50:57.261 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-02-20T07:50:57.263 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-02-20T07:50:57.304 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.061) 0:01:52.800 ***** 2021-02-20T07:50:57.358 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-02-20T07:50:57.359 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:57.399 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.091) 0:01:52.891 ***** 2021-02-20T07:50:57.552 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-02-20T07:50:57.589 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.193) 0:01:53.084 ***** 2021-02-20T07:50:57.596 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-20T07:50:57.597 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-02-20T07:50:57.636 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.047) 0:01:53.132 ***** 2021-02-20T07:50:57.813 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-20T07:50:57.816 INFO:teuthology.task.ansible.out: ok: [smithi106.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T07:50:57.853 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.217) 0:01:53.350 ***** 2021-02-20T07:50:57.925 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-02-20T07:50:57.927 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-02-20T07:50:57.963 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:57 +0000 (0:00:00.109) 0:01:53.459 ***** 2021-02-20T07:50:58.002 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-20T07:50:58.004 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:50:58.041 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.077) 0:01:53.537 ***** 2021-02-20T07:50:58.132 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-02-20T07:50:58.134 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-02-20T07:50:58.170 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.128) 0:01:53.666 ***** 2021-02-20T07:50:58.303 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-02-20T07:50:58.305 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:50:58.341 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.171) 0:01:53.837 ***** 2021-02-20T07:50:58.359 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T07:50:58.360 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:48:50 UTC", "ActiveEnterTimestampMonotonic": "22605236", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target nss-lookup.target basic.target -.mount remote-fs.target sysinit.target systemd-tmpfiles-setup.service systemd-journald.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:49 UTC", "AssertTimestampMonotonic": "22139794", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:49 UTC", "ConditionTimestampMonotonic": "22139794", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T07:50:58.361 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1271", "ExecMainStartTimestamp": "Sat 2021-02-20 07:48:50 UTC", "ExecMainStartTimestampMonotonic": "22605209", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:49 UTC", "InactiveExitTimestampMonotonic": "22140609", "InvocationID": "14e4801f1bb14584813e3a849e16adda", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T07:50:58.363 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1271", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T07:50:58.364 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:48:50 UTC", "StateChangeTimestampMonotonic": "22605236", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:48:50 UTC", "WatchdogTimestampMonotonic": "22605234", "WatchdogUSec": "0"}} 2021-02-20T07:50:58.397 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.055) 0:01:53.893 ***** 2021-02-20T07:50:58.425 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T07:50:58.427 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.462 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.064) 0:01:53.958 ***** 2021-02-20T07:50:58.490 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T07:50:58.492 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.527 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.065) 0:01:54.023 ***** 2021-02-20T07:50:58.567 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T07:50:58.569 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.610 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.082) 0:01:54.106 ***** 2021-02-20T07:50:58.628 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-02-20T07:50:58.630 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:48:56 UTC", "ActiveEnterTimestampMonotonic": "22330008", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice remote-fs.target systemd-journald.socket network.target nss-lookup.target sysinit.target -.mount systemd-tmpfiles-setup.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:55 UTC", "AssertTimestampMonotonic": "21747960", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:55 UTC", "ConditionTimestampMonotonic": "21747959", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-02-20T07:50:58.631 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1247", "ExecMainStartTimestamp": "Sat 2021-02-20 07:48:56 UTC", "ExecMainStartTimestampMonotonic": "22329980", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:55 UTC", "InactiveExitTimestampMonotonic": "21748790", "InvocationID": "4630056b7d6d4ccd834face1372d3ace", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-02-20T07:50:58.634 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1247", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-02-20T07:50:58.636 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:48:56 UTC", "StateChangeTimestampMonotonic": "22330008", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:48:56 UTC", "WatchdogTimestampMonotonic": "22330006", "WatchdogUSec": "0"}} 2021-02-20T07:50:58.669 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.058) 0:01:54.165 ***** 2021-02-20T07:50:58.676 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T07:50:58.678 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.716 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.047) 0:01:54.212 ***** 2021-02-20T07:50:58.723 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-02-20T07:50:58.725 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.765 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.049) 0:01:54.261 ***** 2021-02-20T07:50:58.774 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T07:50:58.775 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.822 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.056) 0:01:54.318 ***** 2021-02-20T07:50:58.830 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-02-20T07:50:58.831 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.872 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.050) 0:01:54.368 ***** 2021-02-20T07:50:58.879 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T07:50:58.881 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.921 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.049) 0:01:54.417 ***** 2021-02-20T07:50:58.929 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-02-20T07:50:58.931 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:58.969 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:58 +0000 (0:00:00.047) 0:01:54.465 ***** 2021-02-20T07:50:58.976 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T07:50:58.979 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.014 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:54.510 ***** 2021-02-20T07:50:59.022 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-02-20T07:50:59.024 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.059 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:54.555 ***** 2021-02-20T07:50:59.066 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-02-20T07:50:59.068 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.105 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:54.601 ***** 2021-02-20T07:50:59.111 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-02-20T07:50:59.113 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.044) 0:01:54.645 ***** 2021-02-20T07:50:59.155 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:50:59.157 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.194 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.044) 0:01:54.690 ***** 2021-02-20T07:50:59.201 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-02-20T07:50:59.203 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.239 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:54.735 ***** 2021-02-20T07:50:59.246 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T07:50:59.248 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.286 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.046) 0:01:54.782 ***** 2021-02-20T07:50:59.303 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-02-20T07:50:59.305 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.340 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.053) 0:01:54.836 ***** 2021-02-20T07:50:59.346 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T07:50:59.348 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.387 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.046) 0:01:54.882 ***** 2021-02-20T07:50:59.396 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.432 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:54.928 ***** 2021-02-20T07:50:59.438 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T07:50:59.440 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.483 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.051) 0:01:54.980 ***** 2021-02-20T07:50:59.490 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:50:59.492 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.529 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.045) 0:01:55.025 ***** 2021-02-20T07:50:59.536 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T07:50:59.538 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.579 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.049) 0:01:55.075 ***** 2021-02-20T07:50:59.588 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-02-20T07:50:59.590 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.627 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.048) 0:01:55.123 ***** 2021-02-20T07:50:59.635 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:50:59.637 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.676 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.048) 0:01:55.172 ***** 2021-02-20T07:50:59.683 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-02-20T07:50:59.685 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.724 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.048) 0:01:55.220 ***** 2021-02-20T07:50:59.730 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:50:59.732 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.767 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.042) 0:01:55.263 ***** 2021-02-20T07:50:59.774 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-02-20T07:50:59.776 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.812 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.044) 0:01:55.308 ***** 2021-02-20T07:50:59.818 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:50:59.820 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.858 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.046) 0:01:55.355 ***** 2021-02-20T07:50:59.866 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T07:50:59.868 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.902 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.043) 0:01:55.398 ***** 2021-02-20T07:50:59.908 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T07:50:59.910 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.945 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.043) 0:01:55.441 ***** 2021-02-20T07:50:59.953 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:50:59.955 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:50:59.999 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:50:59 +0000 (0:00:00.054) 0:01:55.496 ***** 2021-02-20T07:51:00.006 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T07:51:00.008 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.043 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:55.539 ***** 2021-02-20T07:51:00.050 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:51:00.051 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.088 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.044) 0:01:55.584 ***** 2021-02-20T07:51:00.094 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T07:51:00.096 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.133 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.045) 0:01:55.629 ***** 2021-02-20T07:51:00.139 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:51:00.141 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.177 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.044) 0:01:55.673 ***** 2021-02-20T07:51:00.183 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:51:00.185 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.221 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:55.717 ***** 2021-02-20T07:51:00.227 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T07:51:00.229 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.265 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:55.761 ***** 2021-02-20T07:51:00.271 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:51:00.273 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.309 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:55.805 ***** 2021-02-20T07:51:00.317 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T07:51:00.319 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.359 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.050) 0:01:55.855 ***** 2021-02-20T07:51:00.366 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:51:00.368 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.418 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.057) 0:01:55.913 ***** 2021-02-20T07:51:00.423 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-02-20T07:51:00.425 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.462 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.045) 0:01:55.958 ***** 2021-02-20T07:51:00.469 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:51:00.470 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.506 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:56.002 ***** 2021-02-20T07:51:00.512 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:51:00.514 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.551 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.045) 0:01:56.047 ***** 2021-02-20T07:51:00.558 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:51:00.560 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.596 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.044) 0:01:56.092 ***** 2021-02-20T07:51:00.604 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:51:00.606 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.641 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.044) 0:01:56.137 ***** 2021-02-20T07:51:00.648 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:51:00.650 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.690 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.049) 0:01:56.186 ***** 2021-02-20T07:51:00.697 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:51:00.699 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.734 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.043) 0:01:56.230 ***** 2021-02-20T07:51:00.741 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.778 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.044) 0:01:56.274 ***** 2021-02-20T07:51:00.785 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-02-20T07:51:00.787 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.823 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.045) 0:01:56.319 ***** 2021-02-20T07:51:00.852 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-02-20T07:51:00.853 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:00.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.089) 0:01:56.409 ***** 2021-02-20T07:51:00.933 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-20T07:51:00.935 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T07:51:00.974 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:00 +0000 (0:00:00.060) 0:01:56.470 ***** 2021-02-20T07:51:00.982 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-02-20T07:51:00.983 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:01.022 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.048) 0:01:56.518 ***** 2021-02-20T07:51:01.051 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-02-20T07:51:01.053 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:01.091 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.068) 0:01:56.587 ***** 2021-02-20T07:51:01.216 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-02-20T07:51:01.217 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-02-20T07:51:01.260 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.169) 0:01:56.756 ***** 2021-02-20T07:51:01.299 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T07:51:01.300 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T07:51:01.341 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.080) 0:01:56.837 ***** 2021-02-20T07:51:01.379 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T07:51:01.381 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:01.421 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.079) 0:01:56.917 ***** 2021-02-20T07:51:01.567 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-02-20T07:51:01.569 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-02-20T07:51:01.606 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.184) 0:01:57.102 ***** 2021-02-20T07:51:01.632 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-02-20T07:51:01.634 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:01.668 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.062) 0:01:57.164 ***** 2021-02-20T07:51:01.759 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-02-20T07:51:01.764 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T07:51:01.804 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:01 +0000 (0:00:00.135) 0:01:57.300 ***** 2021-02-20T07:51:02.033 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-02-20T07:51:02.035 INFO:teuthology.task.ansible.out: 2021-02-20T07:51:02.077 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.273) 0:01:57.573 ***** 2021-02-20T07:51:02.139 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-02-20T07:51:02.141 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T07:51:02.179 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.101) 0:01:57.675 ***** 2021-02-20T07:51:02.409 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-02-20T07:51:02.453 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.273) 0:01:57.949 ***** 2021-02-20T07:51:02.513 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-02-20T07:51:02.515 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T07:51:02.558 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.104) 0:01:58.054 ***** 2021-02-20T07:51:02.691 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-20T07:51:02.693 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:51:02.728 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.170) 0:01:58.224 ***** 2021-02-20T07:51:02.757 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:02.798 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.070) 0:01:58.294 ***** 2021-02-20T07:51:02.805 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-02-20T07:51:02.807 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-02-20T07:51:02.846 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.048) 0:01:58.342 ***** 2021-02-20T07:51:02.854 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-02-20T07:51:02.856 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:02.894 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.047) 0:01:58.390 ***** 2021-02-20T07:51:02.925 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-20T07:51:02.926 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:02.963 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:02 +0000 (0:00:00.069) 0:01:58.459 ***** 2021-02-20T07:51:02.982 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-02-20T07:51:02.983 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:51:03.027 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.063) 0:01:58.523 ***** 2021-02-20T07:51:03.037 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-02-20T07:51:03.038 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.098 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.071) 0:01:58.594 ***** 2021-02-20T07:51:03.107 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.144 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.046) 0:01:58.640 ***** 2021-02-20T07:51:03.150 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-20T07:51:03.152 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.189 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.045) 0:01:58.685 ***** 2021-02-20T07:51:03.197 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.259 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.069) 0:01:58.755 ***** 2021-02-20T07:51:03.267 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:03.269 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T07:51:03.322 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.063) 0:01:58.818 ***** 2021-02-20T07:51:03.331 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-02-20T07:51:03.333 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.372 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.049) 0:01:58.868 ***** 2021-02-20T07:51:03.389 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-20T07:51:03.391 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.402 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.403 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.440 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.068) 0:01:58.936 ***** 2021-02-20T07:51:03.448 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-20T07:51:03.450 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.450 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-20T07:51:03.451 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.462 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.464 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.474 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-02-20T07:51:03.475 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi106.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.512 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.072) 0:01:59.008 ***** 2021-02-20T07:51:03.519 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.521 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.564 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.051) 0:01:59.060 ***** 2021-02-20T07:51:03.570 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:03.572 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-02-20T07:51:03.609 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.044) 0:01:59.105 ***** 2021-02-20T07:51:03.668 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-02-20T07:51:03.670 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.680 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.691 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.693 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.703 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.705 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.714 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-02-20T07:51:03.726 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.738 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.740 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.750 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.760 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.782 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.793 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.797 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004936", "end": "2021-02-20 07:51:03.764184", "rc": 0, "start": "2021-02-20 07:51:03.759248", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi114.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.836 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.227) 0:01:59.332 ***** 2021-02-20T07:51:03.874 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.038) 0:01:59.370 ***** 2021-02-20T07:51:03.884 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:03.935 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:03 +0000 (0:00:00.061) 0:01:59.431 ***** 2021-02-20T07:51:03.964 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:04.006 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.070) 0:01:59.502 ***** 2021-02-20T07:51:04.036 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-20T07:51:04.038 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:04.073 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.067) 0:01:59.569 ***** 2021-02-20T07:51:04.092 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-02-20T07:51:04.094 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.008051", "end": "2021-02-20 07:51:04.067224", "rc": 0, "start": "2021-02-20 07:51:04.059173", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology-lock"]} 2021-02-20T07:51:04.134 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.060) 0:01:59.630 ***** 2021-02-20T07:51:04.174 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-02-20T07:51:04.176 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:04.215 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.081) 0:01:59.711 ***** 2021-02-20T07:51:04.253 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-02-20T07:51:04.254 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:04.291 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.075) 0:01:59.787 ***** 2021-02-20T07:51:04.319 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-02-20T07:51:04.321 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:04.356 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:04 +0000 (0:00:00.064) 0:01:59.852 ***** 2021-02-20T07:51:05.000 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T07:51:05.002 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:51:05.046 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.690) 0:02:00.542 ***** 2021-02-20T07:51:05.073 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-20T07:51:05.075 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:05.110 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.063) 0:02:00.606 ***** 2021-02-20T07:51:05.172 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T07:51:05.173 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'block_used': 13754889, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340219, 'size_available': 927089537024, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340219, "block_size": 4096, "block_total": 240095108, "block_used": 13754889, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089537024, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T07:51:05.212 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.101) 0:02:00.708 ***** 2021-02-20T07:51:05.229 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-02-20T07:51:05.231 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:51:05.273 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.060) 0:02:00.769 ***** 2021-02-20T07:51:05.301 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-02-20T07:51:05.302 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:05.349 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.076) 0:02:00.845 ***** 2021-02-20T07:51:05.371 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-20T07:51:05.372 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009658", "end": "2021-02-20 07:51:05.345390", "rc": 0, "start": "2021-02-20 07:51:05.335732", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:05.415 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.066) 0:02:00.911 ***** 2021-02-20T07:51:05.423 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-02-20T07:51:05.425 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item={'block_used': 13754888, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340220, 'size_available': 927089541120, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340220, "block_size": 4096, "block_total": 240095108, "block_used": 13754888, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089541120, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}} 2021-02-20T07:51:05.462 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.047) 0:02:00.958 ***** 2021-02-20T07:51:05.469 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-20T07:51:05.471 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'block_used': 13754889, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340219, 'size_available': 927089537024, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340219, "block_size": 4096, "block_total": 240095108, "block_used": 13754889, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089537024, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:05.523 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.060) 0:02:01.019 ***** 2021-02-20T07:51:05.561 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-20T07:51:05.609 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.086) 0:02:01.105 ***** 2021-02-20T07:51:05.618 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-02-20T07:51:05.620 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011073", "end": "2021-02-20 07:51:05.571828", "rc": 0, "start": "2021-02-20 07:51:05.560755", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:05.662 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.052) 0:02:01.158 ***** 2021-02-20T07:51:05.672 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T07:51:05.673 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:05.727 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-02-20T07:51:05.729 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'block_used': 13754888, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340220, 'size_available': 927089541120, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340220, "block_size": 4096, "block_total": 240095108, "block_used": 13754888, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089541120, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:05.767 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.105) 0:02:01.263 ***** 2021-02-20T07:51:05.798 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-02-20T07:51:05.836 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:05 +0000 (0:00:00.068) 0:02:01.332 ***** 2021-02-20T07:51:05.887 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-02-20T07:51:05.888 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:07.764 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJGeH28ySJbRAAKcohPtmDzKM6MIrOz1IQ'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.732235", "end": "2021-02-20 07:51:07.722891", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJGeH28ySJbRAAKcohPtmDzKM6MIrOz1IQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:05.990656", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:07.775 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:07.787 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:07.851 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu9M8uXn9ky77oJ4Zov6Y0ljF1K6QadIXL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:02.068362", "end": "2021-02-20 07:51:07.815204", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu9M8uXn9ky77oJ4Zov6Y0ljF1K6QadIXL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:05.746842", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:07.863 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:07.864 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:09.051 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJ2jRh2X6p2s8fk50gVoFVHJybk1otWlLv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.141885", "end": "2021-02-20 07:51:09.009305", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJ2jRh2X6p2s8fk50gVoFVHJybk1otWlLv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:07.867420", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:09.052 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYVM', 'wwn-0x5000c50091e2f173'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYVM-part1', 'wwn-0x5000c50091e2f173-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e2f173', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QYVM', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYVM", "wwn-0x5000c50091e2f173"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYVM-part1", "wwn-0x5000c50091e2f173-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QYVM", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2f173"}}, "skip_reason": 2021-02-20T07:51:09.054 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T07:51:09.137 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMuktMlFIpCekJYMX9S8LWa1jM96UIHM72w'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.155053", "end": "2021-02-20 07:51:09.108196", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMuktMlFIpCekJYMX9S8LWa1jM96UIHM72w"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:07.953143", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:09.159 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ', 'wwn-0x5000c50091e1aea6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', 'wwn-0x5000c50091e1aea6-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1aea6', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QHPJ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, "skip_reason": 2021-02-20T07:51:09.161 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T07:51:10.368 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-zvuxwi-LL6P-40yC-mJXM-6TA2-Mx5Q-Wop8Tq', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400WB400BGN', 'nvme-nvme.8086-50484654363230343030574234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.192258", "end": "2021-02-20 07:51:10.332349", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zvuxwi-LL6P-40yC-mJXM-6TA2-Mx5Q-Wop8Tq", "nvme-INTEL_SSDPEDMD400G4_PHFT620400WB400BGN", "nvme-nvme.8086-50484654363230343030574234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:09.140091", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-02-20T07:51:10.370 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:10.380 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:10.394 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:10.476 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-xBALIm-3Woy-T8lg-zp0J-4eh8-cl7I-Hzyn8j', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', 'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.192352", "end": "2021-02-20 07:51:10.440743", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-xBALIm-3Woy-T8lg-zp0J-4eh8-cl7I-Hzyn8j", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:09.248391", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-02-20T07:51:10.478 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:10.488 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:10.489 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:11.694 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJkdnDBVCbZobaWB0BS9z2oB7McmeGHtof'], 'uuids': ['5f1aa4aa-ba5d-401c-b8da-b103adecec09']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.180489", "end": "2021-02-20 07:51:11.651078", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJkdnDBVCbZobaWB0BS9z2oB7McmeGHtof"], "labels": [], "masters": [], "uuids": ["5f1aa4aa-ba5d-401c-b8da-b103adecec09"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:10.470589", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:11.705 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:11.707 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:11.783 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu0fVk2xngMaX8s7mafIOEtwF6GPw9Z8jZ'], 'uuids': ['58d24308-83ac-4a75-afc1-0e4251ba7c8f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.174031", "end": "2021-02-20 07:51:11.753140", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu0fVk2xngMaX8s7mafIOEtwF6GPw9Z8jZ"], "labels": [], "masters": [], "uuids": ["58d24308-83ac-4a75-afc1-0e4251ba7c8f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:10.579109", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:11.805 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:11.817 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:12.978 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJb92ba6jpumVvLukyGDAUrCO9i57Zi0gA'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.138465", "end": "2021-02-20 07:51:12.934594", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJb92ba6jpumVvLukyGDAUrCO9i57Zi0gA"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:11.796129", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:12.992 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:13.068 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMucXzyQdxdEWxaXzyFqGWMPvM2v0Zvuats'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.147043", "end": "2021-02-20 07:51:13.035276", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMucXzyQdxdEWxaXzyFqGWMPvM2v0Zvuats"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:11.888233", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:13.084 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.274 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJVNKfwS3Sdy5F0U0QwYTgOXOw57419stN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.151816", "end": "2021-02-20 07:51:14.228114", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJVNKfwS3Sdy5F0U0QwYTgOXOw57419stN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:13.076298", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:14.330 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:14 +0000 (0:00:08.494) 0:02:09.826 ***** 2021-02-20T07:51:14.331 INFO:teuthology.task.ansible.out: 2021-02-20T07:51:14.344 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu6kd8EPKTAb41igyBelY3A8iOC0Oa5HYm'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.137995", "end": "2021-02-20 07:51:14.309491", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu6kd8EPKTAb41igyBelY3A8iOC0Oa5HYm"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:13.171496", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-02-20T07:51:14.396 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:14 +0000 (0:00:00.065) 0:02:09.892 ***** 2021-02-20T07:51:14.408 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-02-20T07:51:14.410 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.453 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.709 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJGeH28ySJbRAAKcohPtmDzKM6MIrOz1IQ'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.159170", "end": "2021-02-20 07:51:14.673724", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJGeH28ySJbRAAKcohPtmDzKM6MIrOz1IQ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:14.514554", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154565 s, 746 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154565 s, 746 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:14.723 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.737 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.750 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu9M8uXn9ky77oJ4Zov6Y0ljF1K6QadIXL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.153846", "end": "2021-02-20 07:51:14.715413", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu9M8uXn9ky77oJ4Zov6Y0ljF1K6QadIXL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:14.561567", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149187 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149187 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:14.763 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:14.786 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.031 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJ2jRh2X6p2s8fk50gVoFVHJybk1otWlLv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.154985", "end": "2021-02-20 07:51:14.994147", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJ2jRh2X6p2s8fk50gVoFVHJybk1otWlLv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:14.839162", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150507 s, 766 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150507 s, 766 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.044 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYVM', 'wwn-0x5000c50091e2f173'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYVM-part1', 'wwn-0x5000c50091e2f173-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e2f173', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QYVM', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYVM", "wwn-0x5000c50091e2f173"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYVM-part1", "wwn-0x5000c50091e2f173-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QYVM", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2f173"}}, "skip_reason": 2021-02-20T07:51:15.046 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T07:51:15.067 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMuktMlFIpCekJYMX9S8LWa1jM96UIHM72w'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.159337", "end": "2021-02-20 07:51:15.028706", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMuktMlFIpCekJYMX9S8LWa1jM96UIHM72w"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:14.869369", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156244 s, 738 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156244 s, 738 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.079 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ', 'wwn-0x5000c50091e1aea6'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', 'wwn-0x5000c50091e1aea6-part1'], 'uuids': ['602a9f0b-ccf6-4d5d-877a-542e0952c07a']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1aea6', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QHPJ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["602a9f0b-ccf6-4d5d-877a-542e0952c07a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, "skip_reason": 2021-02-20T07:51:15.081 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-02-20T07:51:15.220 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-zvuxwi-LL6P-40yC-mJXM-6TA2-Mx5Q-Wop8Tq', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400WB400BGN', 'nvme-nvme.8086-50484654363230343030574234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.055692", "end": "2021-02-20 07:51:15.184044", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zvuxwi-LL6P-40yC-mJXM-6TA2-Mx5Q-Wop8Tq", "nvme-INTEL_SSDPEDMD400G4_PHFT620400WB400BGN", "nvme-nvme.8086-50484654363230343030574234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.128352", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0509182 s, 2.3 GB/s", "std 2021-02-20T07:51:15.222 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0509182 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.232 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.244 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.273 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-xBALIm-3Woy-T8lg-zp0J-4eh8-cl7I-Hzyn8j', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', 'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.056094", "end": "2021-02-20 07:51:15.224434", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-xBALIm-3Woy-T8lg-zp0J-4eh8-cl7I-Hzyn8j", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.168340", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0512445 s, 2.3 GB/s", "std 2021-02-20T07:51:15.275 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0512445 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.280 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.293 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.527 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJkdnDBVCbZobaWB0BS9z2oB7McmeGHtof'], 'uuids': ['5f1aa4aa-ba5d-401c-b8da-b103adecec09']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.150765", "end": "2021-02-20 07:51:15.485641", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJkdnDBVCbZobaWB0BS9z2oB7McmeGHtof"], "labels": [], "masters": [], "uuids": ["5f1aa4aa-ba5d-401c-b8da-b103adecec09"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.334876", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14815 s, 779 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14815 s, 779 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.529 INFO:teuthology.task.ansible.out: 2021-02-20T07:51:15.539 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.541 INFO:teuthology.task.ansible.out: skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.563 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu0fVk2xngMaX8s7mafIOEtwF6GPw9Z8jZ'], 'uuids': ['58d24308-83ac-4a75-afc1-0e4251ba7c8f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.149211", "end": "2021-02-20 07:51:15.529172", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu0fVk2xngMaX8s7mafIOEtwF6GPw9Z8jZ"], "labels": [], "masters": [], "uuids": ["58d24308-83ac-4a75-afc1-0e4251ba7c8f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.379961", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.146571 s, 787 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.146571 s, 787 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.576 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.588 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.833 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJb92ba6jpumVvLukyGDAUrCO9i57Zi0gA'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.160034", "end": "2021-02-20 07:51:15.790146", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJb92ba6jpumVvLukyGDAUrCO9i57Zi0gA"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.630112", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155265 s, 743 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155265 s, 743 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.838 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:15.889 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMucXzyQdxdEWxaXzyFqGWMPvM2v0Zvuats'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.171365", "end": "2021-02-20 07:51:15.848786", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMucXzyQdxdEWxaXzyFqGWMPvM2v0Zvuats"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.677421", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.166643 s, 692 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.166643 s, 692 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:15.900 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:16.120 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJVNKfwS3Sdy5F0U0QwYTgOXOw57419stN'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.161672", "end": "2021-02-20 07:51:16.083975", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TzhdvnB6dxzie3rWnO1AUQLEqihI6vAJVNKfwS3Sdy5F0U0QwYTgOXOw57419stN"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:15.922303", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157027 s, 735 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157027 s, 735 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.184 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:01.787) 0:02:11.680 ***** 2021-02-20T07:51:16.203 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu6kd8EPKTAb41igyBelY3A8iOC0Oa5HYm'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.152715", "end": "2021-02-20 07:51:16.154583", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-emIl0ISd2A02KyyZqgEEEh7ARx9U5wMu6kd8EPKTAb41igyBelY3A8iOC0Oa5HYm"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-02-20 07:51:16.001868", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.148128 s, 779 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.148128 s, 779 MB/s"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.255 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.071) 0:02:11.751 ***** 2021-02-20T07:51:16.446 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-02-20T07:51:16.448 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.128220", "end": "2021-02-20 07:51:16.408288", "rc": 0, "start": "2021-02-20 07:51:16.280068", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.488 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.233) 0:02:11.984 ***** 2021-02-20T07:51:16.521 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.120267", "end": "2021-02-20 07:51:16.488523", "rc": 0, "start": "2021-02-20 07:51:16.368256", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.564 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.075) 0:02:12.060 ***** 2021-02-20T07:51:16.684 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-02-20T07:51:16.688 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.045912", "end": "2021-02-20 07:51:16.641845", "rc": 0, "start": "2021-02-20 07:51:16.595933", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.745 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.179) 0:02:12.240 ***** 2021-02-20T07:51:16.755 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.039820", "end": "2021-02-20 07:51:16.709204", "rc": 0, "start": "2021-02-20 07:51:16.669384", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:16.807 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.062) 0:02:12.303 ***** 2021-02-20T07:51:16.816 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-02-20T07:51:16.859 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.052) 0:02:12.355 ***** 2021-02-20T07:51:16.906 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.046) 0:02:12.402 ***** 2021-02-20T07:51:16.913 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-02-20T07:51:16.916 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:16.953 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:16 +0000 (0:00:00.046) 0:02:12.449 ***** 2021-02-20T07:51:16.960 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.010 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.057) 0:02:12.506 ***** 2021-02-20T07:51:17.020 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.074 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.063) 0:02:12.570 ***** 2021-02-20T07:51:17.082 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.138 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.064) 0:02:12.634 ***** 2021-02-20T07:51:17.151 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T07:51:17.153 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.203 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.064) 0:02:12.699 ***** 2021-02-20T07:51:17.211 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.260 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.057) 0:02:12.756 ***** 2021-02-20T07:51:17.268 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-02-20T07:51:17.270 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.312 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.050) 0:02:12.807 ***** 2021-02-20T07:51:17.324 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.368 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.057) 0:02:12.864 ***** 2021-02-20T07:51:17.451 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-02-20T07:51:17.454 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:51:17.511 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.143) 0:02:13.007 ***** 2021-02-20T07:51:17.519 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-02-20T07:51:17.556 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.044) 0:02:13.052 ***** 2021-02-20T07:51:17.574 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi114.front.sepia.ceph.com] => (item={'block_used': 13754888, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340220, 'size_available': 927089541120, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340220, "block_size": 4096, "block_total": 240095108, "block_used": 13754888, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089541120, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.621 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.064) 0:02:13.117 ***** 2021-02-20T07:51:17.639 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'block_used': 13754889, 'uuid': '602a9f0b-ccf6-4d5d-877a-542e0952c07a', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226340219, 'size_available': 927089537024, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149545, 'block_size': 4096, 'inode_available': 60905431}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226340219, "block_size": 4096, "block_total": 240095108, "block_used": 13754889, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905431, "inode_total": 61054976, "inode_used": 149545, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927089537024, "size_total": 983429562368, "uuid": "602a9f0b-ccf6-4d5d-877a-542e0952c07a"}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.688 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.066) 0:02:13.184 ***** 2021-02-20T07:51:17.698 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.058) 0:02:13.243 ***** 2021-02-20T07:51:17.754 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.793 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.046) 0:02:13.289 ***** 2021-02-20T07:51:17.806 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.840 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.046) 0:02:13.336 ***** 2021-02-20T07:51:17.862 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.901 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.057) 0:02:13.393 ***** 2021-02-20T07:51:17.904 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:17.906 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:17.953 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:17 +0000 (0:00:00.055) 0:02:13.449 ***** 2021-02-20T07:51:17.966 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.013 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.059) 0:02:13.508 ***** 2021-02-20T07:51:18.021 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-02-20T07:51:18.022 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.067 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.054) 0:02:13.563 ***** 2021-02-20T07:51:18.073 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.135 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.064) 0:02:13.627 ***** 2021-02-20T07:51:18.140 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-02-20T07:51:18.142 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.191 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.059) 0:02:13.687 ***** 2021-02-20T07:51:18.203 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.250 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.059) 0:02:13.746 ***** 2021-02-20T07:51:18.259 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-02-20T07:51:18.260 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.303 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.053) 0:02:13.800 ***** 2021-02-20T07:51:18.312 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:18.352 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.048) 0:02:13.848 ***** 2021-02-20T07:51:18.942 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-02-20T07:51:18.944 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T07:51:18.945 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-02-20T07:51:18.982 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:18 +0000 (0:00:00.629) 0:02:14.478 ***** 2021-02-20T07:51:19.019 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:19 +0000 (0:00:00.036) 0:02:14.515 ***** 2021-02-20T07:51:19.570 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-02-20T07:51:19.571 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:19.592 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:19.804 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:19.805 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.019 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.052 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.272 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.284 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.495 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:20 +0000 (0:00:01.514) 0:02:16.029 ***** 2021-02-20T07:51:20.540 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-02-20T07:51:20.582 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:20 +0000 (0:00:00.049) 0:02:16.078 ***** 2021-02-20T07:51:20.663 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-02-20T07:51:20.665 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T07:51:20.705 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:20 +0000 (0:00:00.122) 0:02:16.201 ***** 2021-02-20T07:51:20.726 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-02-20T07:51:20.767 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:20 +0000 (0:00:00.061) 0:02:16.263 ***** 2021-02-20T07:51:20.849 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-20T07:51:20.850 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:20.924 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:20.967 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:21.063 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:21.074 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:21.180 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:21.182 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-02-20T07:51:21.204 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.247 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.479) 0:02:16.743 ***** 2021-02-20T07:51:21.275 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-20T07:51:21.277 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.311 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.064) 0:02:16.807 ***** 2021-02-20T07:51:21.317 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-02-20T07:51:21.319 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} skipping: [smithi106.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.361 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.049) 0:02:16.857 ***** 2021-02-20T07:51:21.368 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-20T07:51:21.369 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.405 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.044) 0:02:16.901 ***** 2021-02-20T07:51:21.412 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-02-20T07:51:21.413 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.452 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.046) 0:02:16.948 ***** 2021-02-20T07:51:21.491 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-02-20T07:51:21.493 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:21.529 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.076) 0:02:17.025 ***** 2021-02-20T07:51:21.546 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-20T07:51:21.547 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:21.584 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.055) 0:02:17.080 ***** 2021-02-20T07:51:21.622 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-20T07:51:21.624 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T07:51:21.663 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.078) 0:02:17.159 ***** 2021-02-20T07:51:21.670 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-02-20T07:51:21.672 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:21.709 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.046) 0:02:17.205 ***** 2021-02-20T07:51:21.746 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-02-20T07:51:21.748 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-02-20T07:51:21.781 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:21 +0000 (0:00:00.072) 0:02:17.278 ***** 2021-02-20T07:51:22.141 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi114.front.sepia.ceph.com] => {"changed": true} 2021-02-20T07:51:22.179 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.397) 0:02:17.675 ***** 2021-02-20T07:51:22.186 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true} 2021-02-20T07:51:22.226 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.046) 0:02:17.722 ***** 2021-02-20T07:51:22.675 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-02-20T07:51:22.677 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-02-20T07:51:22.721 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.495) 0:02:18.217 ***** 2021-02-20T07:51:22.767 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.045) 0:02:18.263 ***** 2021-02-20T07:51:22.773 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-02-20T07:51:22.775 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:22.816 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.045) 0:02:18.308 ***** 2021-02-20T07:51:22.821 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:22.863 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.050) 0:02:18.359 ***** 2021-02-20T07:51:22.869 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-02-20T07:51:22.871 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:22.907 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.044) 0:02:18.403 ***** 2021-02-20T07:51:22.913 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:22.952 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.044) 0:02:18.448 ***** 2021-02-20T07:51:22.958 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T07:51:22.960 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:22.995 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:22 +0000 (0:00:00.043) 0:02:18.491 ***** 2021-02-20T07:51:23.002 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:23.039 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.044) 0:02:18.535 ***** 2021-02-20T07:51:23.045 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-02-20T07:51:23.047 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:23.085 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.045) 0:02:18.581 ***** 2021-02-20T07:51:23.093 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:23.130 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.045) 0:02:18.626 ***** 2021-02-20T07:51:23.241 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-02-20T07:51:23.243 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T07:51:23.282 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.152) 0:02:18.778 ***** 2021-02-20T07:51:23.291 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T07:51:23.329 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.046) 0:02:18.825 ***** 2021-02-20T07:51:23.431 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:23.471 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.141) 0:02:18.967 ***** 2021-02-20T07:51:23.480 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:23.531 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.059) 0:02:19.027 ***** 2021-02-20T07:51:23.809 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-02-20T07:51:23.811 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T07:51:23.847 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.316) 0:02:19.343 ***** 2021-02-20T07:51:23.856 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-02-20T07:51:23.894 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:23 +0000 (0:00:00.046) 0:02:19.390 ***** 2021-02-20T07:51:24.183 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-02-20T07:51:24.185 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T07:51:24.221 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:24 +0000 (0:00:00.326) 0:02:19.717 ***** 2021-02-20T07:51:24.239 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-02-20T07:51:24.278 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:24 +0000 (0:00:00.057) 0:02:19.774 ***** 2021-02-20T07:51:25.131 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:51:25.181 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.902) 0:02:20.677 ***** 2021-02-20T07:51:25.210 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:51:25.250 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.069) 0:02:20.746 ***** 2021-02-20T07:51:25.387 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-02-20T07:51:25.389 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047218", "end": "2021-02-20 07:51:25.350838", "rc": 0, "start": "2021-02-20 07:51:25.303620", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T07:51:25.427 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.176) 0:02:20.923 ***** 2021-02-20T07:51:25.436 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.043056", "end": "2021-02-20 07:51:25.400988", "rc": 0, "start": "2021-02-20 07:51:25.357932", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-02-20T07:51:25.479 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.051) 0:02:20.975 ***** 2021-02-20T07:51:25.485 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-02-20T07:51:25.487 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:25.532 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.052) 0:02:21.028 ***** 2021-02-20T07:51:25.541 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:25.595 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.062) 0:02:21.091 ***** 2021-02-20T07:51:25.602 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-02-20T07:51:25.603 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:25.643 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.047) 0:02:21.139 ***** 2021-02-20T07:51:25.650 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:25.691 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:25 +0000 (0:00:00.047) 0:02:21.187 ***** 2021-02-20T07:51:26.562 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-02-20T07:51:26.564 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:51:26.602 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:26 +0000 (0:00:00.911) 0:02:22.098 ***** 2021-02-20T07:51:26.611 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:51:26.648 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:26 +0000 (0:00:00.046) 0:02:22.144 ***** 2021-02-20T07:51:26.654 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-02-20T07:51:26.656 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:26.692 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:26 +0000 (0:00:00.044) 0:02:22.188 ***** 2021-02-20T07:51:26.700 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:26.738 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:26 +0000 (0:00:00.045) 0:02:22.234 ***** 2021-02-20T07:51:27.004 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-02-20T07:51:27.006 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T07:51:27.048 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.309) 0:02:22.544 ***** 2021-02-20T07:51:27.056 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-02-20T07:51:27.095 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.047) 0:02:22.591 ***** 2021-02-20T07:51:27.112 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.149 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.053) 0:02:22.645 ***** 2021-02-20T07:51:27.156 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.197 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.047) 0:02:22.693 ***** 2021-02-20T07:51:27.343 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-02-20T07:51:27.344 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:49:01 UTC", "ActiveEnterTimestampMonotonic": "27909139", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice systemd-tmpfiles-setup.service -.mount basic.target sysinit.target systemd-journald.socket network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:55 UTC", "AssertTimestampMonotonic": "21364180", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:55 UTC", "ConditionTimestampMonotonic": "21364179", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T07:51:27.346 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1749", "ExecMainStartTimestamp": "Sat 2021-02-20 07:49:01 UTC", "ExecMainStartTimestampMonotonic": "27909126", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:55 UTC", "InactiveExitTimestampMonotonic": "21365811", "InvocationID": "3e2138f8bb0f4ed99e6d1ec62e52227b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T07:51:27.348 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1749", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:49:01 UTC", "S 2021-02-20T07:51:27.349 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "27909139", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:49:01 UTC", "WatchdogTimestampMonotonic": "27909138", "WatchdogUSec": "0"}} 2021-02-20T07:51:27.386 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.188) 0:02:22.882 ***** 2021-02-20T07:51:27.405 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sat 2021-02-20 07:48:56 UTC", "ActiveEnterTimestampMonotonic": "28458181", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target systemd-tmpfiles-setup.service systemd-journald.socket -.mount basic.target sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sat 2021-02-20 07:48:49 UTC", "AssertTimestampMonotonic": "21792157", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sat 2021-02-20 07:48:49 UTC", "ConditionTimestampMonotonic": "21792156", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-02-20T07:51:27.407 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1701", "ExecMainStartTimestamp": "Sat 2021-02-20 07:48:56 UTC", "ExecMainStartTimestampMonotonic": "28458153", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sat 2021-02-20 07:48:49 UTC", "InactiveExitTimestampMonotonic": "21793837", "InvocationID": "441d7a16465547d8848c938b123388fd", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-02-20T07:51:27.409 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1701", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:48:56 UTC", "S 2021-02-20T07:51:27.410 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "28458181", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Sat 2021-02-20 07:48:56 UTC", "WatchdogTimestampMonotonic": "28458179", "WatchdogUSec": "0"}} 2021-02-20T07:51:27.443 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.057) 0:02:22.939 ***** 2021-02-20T07:51:27.450 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:27.452 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T07:51:27.488 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.045) 0:02:22.984 ***** 2021-02-20T07:51:27.497 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-02-20T07:51:27.533 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.044) 0:02:23.029 ***** 2021-02-20T07:51:27.540 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:27.541 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.578 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.045) 0:02:23.074 ***** 2021-02-20T07:51:27.586 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.629 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.051) 0:02:23.126 ***** 2021-02-20T07:51:27.636 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-02-20T07:51:27.638 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.676 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.046) 0:02:23.172 ***** 2021-02-20T07:51:27.682 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:27.725 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:27 +0000 (0:00:00.048) 0:02:23.221 ***** 2021-02-20T07:51:28.552 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-02-20T07:51:28.553 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:51:28.592 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.867) 0:02:24.088 ***** 2021-02-20T07:51:28.599 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:51:28.637 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.045) 0:02:24.133 ***** 2021-02-20T07:51:28.645 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-02-20T07:51:28.646 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:28.682 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.045) 0:02:24.178 ***** 2021-02-20T07:51:28.691 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:28.733 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.050) 0:02:24.229 ***** 2021-02-20T07:51:28.813 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-02-20T07:51:28.815 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T07:51:28.851 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.118) 0:02:24.347 ***** 2021-02-20T07:51:28.879 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-02-20T07:51:28.917 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:28 +0000 (0:00:00.065) 0:02:24.413 ***** 2021-02-20T07:51:29.169 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T07:51:29.205 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:29 +0000 (0:00:00.288) 0:02:24.701 ***** 2021-02-20T07:51:29.246 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-02-20T07:51:29.284 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:29 +0000 (0:00:00.078) 0:02:24.780 ***** 2021-02-20T07:51:30.236 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-02-20T07:51:30.239 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T07:51:30.281 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.997) 0:02:25.777 ***** 2021-02-20T07:51:30.289 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-02-20T07:51:30.342 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.054) 0:02:25.831 ***** 2021-02-20T07:51:30.781 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-02-20T07:51:30.783 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false} 2021-02-20T07:51:30.828 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.492) 0:02:26.324 ***** 2021-02-20T07:51:30.856 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false} 2021-02-20T07:51:30.858 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-02-20T07:51:30.899 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.071) 0:02:26.395 ***** 2021-02-20T07:51:30.951 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.052) 0:02:26.447 ***** 2021-02-20T07:51:30.960 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-02-20T07:51:31.000 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:30 +0000 (0:00:00.048) 0:02:26.496 ***** 2021-02-20T07:51:31.104 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-02-20T07:51:31.106 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613807116.6202376, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613807116.6162376, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613807116.6162376, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T07:51:31.147 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.146) 0:02:26.643 ***** 2021-02-20T07:51:31.154 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1613807118.8375428, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1613807118.8335428, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 746, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1613807118.8335428, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3375002756", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-02-20T07:51:31.193 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.046) 0:02:26.689 ***** 2021-02-20T07:51:31.541 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T07:51:31.543 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-02-20T07:51:31.583 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.389) 0:02:27.079 ***** 2021-02-20T07:51:31.629 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.045) 0:02:27.125 ***** 2021-02-20T07:51:31.636 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-02-20T07:51:31.638 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T07:51:31.673 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.043) 0:02:27.169 ***** 2021-02-20T07:51:31.680 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-02-20T07:51:31.717 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.043) 0:02:27.213 ***** 2021-02-20T07:51:31.724 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-02-20T07:51:31.726 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:31.770 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.052) 0:02:27.266 ***** 2021-02-20T07:51:31.781 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:31.823 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.053) 0:02:27.319 ***** 2021-02-20T07:51:31.908 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-02-20T07:51:31.910 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T07:51:31.961 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:31 +0000 (0:00:00.137) 0:02:27.457 ***** 2021-02-20T07:51:31.981 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-02-20T07:51:32.028 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.067) 0:02:27.524 ***** 2021-02-20T07:51:32.196 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T07:51:32.197 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T07:51:32.233 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.204) 0:02:27.729 ***** 2021-02-20T07:51:32.260 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-20T07:51:32.262 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.296 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.063) 0:02:27.792 ***** 2021-02-20T07:51:32.314 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-02-20T07:51:32.315 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-02-20T07:51:32.353 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.056) 0:02:27.850 ***** 2021-02-20T07:51:32.360 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T07:51:32.362 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.363 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.374 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.410 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.056) 0:02:27.906 ***** 2021-02-20T07:51:32.417 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-02-20T07:51:32.419 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.465 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.054) 0:02:27.961 ***** 2021-02-20T07:51:32.472 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-20T07:51:32.474 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.474 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.475 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditiona 2021-02-20T07:51:32.476 INFO:teuthology.task.ansible.out:l result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.519 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.054) 0:02:28.015 ***** 2021-02-20T07:51:32.526 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-02-20T07:51:32.527 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.529 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.529 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.573 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.053) 0:02:28.069 ***** 2021-02-20T07:51:32.579 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-02-20T07:51:32.581 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.623 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.049) 0:02:28.119 ***** 2021-02-20T07:51:32.637 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-02-20T07:51:32.639 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.641 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi114.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.653 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.654 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.696 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.072) 0:02:28.192 ***** 2021-02-20T07:51:32.705 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.746 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.050) 0:02:28.243 ***** 2021-02-20T07:51:32.755 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.048) 0:02:28.291 ***** 2021-02-20T07:51:32.802 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T07:51:32.803 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.849 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.053) 0:02:28.345 ***** 2021-02-20T07:51:32.863 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-02-20T07:51:32.865 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.903 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.053) 0:02:28.399 ***** 2021-02-20T07:51:32.912 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-20T07:51:32.913 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.950 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.047) 0:02:28.446 ***** 2021-02-20T07:51:32.957 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-02-20T07:51:32.959 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:32.999 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:32 +0000 (0:00:00.048) 0:02:28.495 ***** 2021-02-20T07:51:33.007 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-20T07:51:33.009 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.050 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.051) 0:02:28.546 ***** 2021-02-20T07:51:33.057 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-02-20T07:51:33.059 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.098 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.048) 0:02:28.594 ***** 2021-02-20T07:51:33.107 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T07:51:33.109 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.153 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.054) 0:02:28.649 ***** 2021-02-20T07:51:33.160 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-02-20T07:51:33.162 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.203 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.050) 0:02:28.699 ***** 2021-02-20T07:51:33.210 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T07:51:33.213 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.251 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.048) 0:02:28.747 ***** 2021-02-20T07:51:33.257 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T07:51:33.260 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.300 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.048) 0:02:28.796 ***** 2021-02-20T07:51:33.306 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-02-20T07:51:33.308 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.345 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.045) 0:02:28.841 ***** 2021-02-20T07:51:33.352 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-02-20T07:51:33.354 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.397 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.051) 0:02:28.893 ***** 2021-02-20T07:51:33.425 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:33.460 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:33 +0000 (0:00:00.063) 0:02:28.956 ***** 2021-02-20T07:51:33.521 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T07:51:33.522 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011083", "end": "2021-02-20 07:51:33.465532", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 07:51:33.454449", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.616 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010985", "end": "2021-02-20 07:51:33.573268", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-02-20 07:51:33.562283", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.638 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.004797", "end": "2021-02-20 07:51:33.591715", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 07:51:33.586918", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.726 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005451", "end": "2021-02-20 07:51:33.691231", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-02-20 07:51:33.685780", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.747 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.003588", "end": "2021-02-20 07:51:33.701408", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 07:51:33.697820", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.844 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005438", "end": "2021-02-20 07:51:33.810804", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-02-20 07:51:33.805366", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:33.981 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.022275", "end": "2021-02-20 07:51:33.944045", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 07:51:33.921770", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:34.020 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.559) 0:02:29.516 ***** 2021-02-20T07:51:34.049 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T07:51:34.112 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.091) 0:02:29.608 ***** 2021-02-20T07:51:34.179 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-02-20T07:51:34.182 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.318856", "end": "2021-02-20 07:51:34.125673", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-02-20 07:51:33.806817", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:34.221 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.109) 0:02:29.717 ***** 2021-02-20T07:51:34.250 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-02-20T07:51:34.251 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-02-20T07:51:34.288 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.066) 0:02:29.784 ***** 2021-02-20T07:51:34.421 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-02-20T07:51:34.423 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-02-20T07:51:34.536 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-02-20T07:51:34.575 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.287) 0:02:30.071 ***** 2021-02-20T07:51:34.614 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:34 +0000 (0:00:00.038) 0:02:30.110 ***** 2021-02-20T07:51:35.260 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-02-20T07:51:35.261 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] 2021-02-20T07:51:35.367 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.753) 0:02:30.863 ***** 2021-02-20T07:51:35.381 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] 2021-02-20T07:51:35.429 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:35.430 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T07:51:35.469 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.101) 0:02:30.965 ***** 2021-02-20T07:51:35.509 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.040) 0:02:31.005 ***** 2021-02-20T07:51:35.516 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:35.518 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:35.558 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.048) 0:02:31.054 ***** 2021-02-20T07:51:35.566 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-02-20T07:51:35.612 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.054) 0:02:31.108 ***** 2021-02-20T07:51:35.620 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T07:51:35.622 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T07:51:35.663 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.050) 0:02:31.159 ***** 2021-02-20T07:51:35.671 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:35.674 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:35.711 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.047) 0:02:31.207 ***** 2021-02-20T07:51:35.751 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-02-20T07:51:35.752 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-02-20T07:51:35.795 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:35 +0000 (0:00:00.084) 0:02:31.291 ***** 2021-02-20T07:51:36.510 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** ok: [smithi106.front.sepia.ceph.com] => {"cache_update_time": 1613807139, "cache_updated": false, "changed": false} 2021-02-20T07:51:36.550 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.755) 0:02:32.047 ***** 2021-02-20T07:51:36.578 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:36.580 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:36.620 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.069) 0:02:32.116 ***** 2021-02-20T07:51:36.658 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:36.659 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T07:51:36.698 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.078) 0:02:32.194 ***** 2021-02-20T07:51:36.718 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-02-20T07:51:36.721 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"cache_update_time": 1613807181, "cache_updated": false, "changed": false} 2021-02-20T07:51:36.762 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.063) 0:02:32.258 ***** 2021-02-20T07:51:36.793 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:36.795 INFO:teuthology.task.ansible.out:skipping: [smithi114.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-02-20T07:51:36.833 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.070) 0:02:32.329 ***** 2021-02-20T07:51:36.851 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-20T07:51:36.853 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:36.873 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-02-20T07:51:36.875 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-02-20T07:51:36.913 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:36 +0000 (0:00:00.080) 0:02:32.409 ***** 2021-02-20T07:51:36.974 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-02-20T07:51:36.976 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:37.029 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:37 +0000 (0:00:00.116) 0:02:32.525 ***** 2021-02-20T07:51:37.068 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:37.184 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-02-20T07:51:37.228 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:37 +0000 (0:00:00.199) 0:02:32.724 ***** 2021-02-20T07:51:37.413 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-02-20T07:51:37.415 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807497.080854-60794240783969/source", "state": "file", "uid": 0} 2021-02-20T07:51:37.594 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807497.2723958-271797797022725/source", "state": "file", "uid": 0} 2021-02-20T07:51:37.736 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807497.4195297-102268847135544/source", "state": "file", "uid": 0} 2021-02-20T07:51:37.782 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:37 +0000 (0:00:00.553) 0:02:33.278 ***** 2021-02-20T07:51:37.920 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807497.6038618-229094806494304/source", "state": "file", "uid": 0} 2021-02-20T07:51:37.962 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:37 +0000 (0:00:00.179) 0:02:33.458 ***** 2021-02-20T07:51:41.457 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-20T07:51:41.459 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-V7lkGf-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-V7lkGf-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T07:51:41.461 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T07:51:41.499 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:41 +0000 (0:00:03.536) 0:02:36.995 ***** 2021-02-20T07:51:41.633 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-02-20T07:51:41.635 INFO:teuthology.task.ansible.out:ok: [smithi114.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-20T07:51:41.673 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:41 +0000 (0:00:00.174) 0:02:37.169 ***** 2021-02-20T07:51:41.680 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-02-20T07:51:41.682 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-bGtW3L-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-bGtW3L-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-02-20T07:51:41.684 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-02-20T07:51:41.720 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:41 +0000 (0:00:00.046) 0:02:37.216 ***** 2021-02-20T07:51:41.843 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-02-20T07:51:41.845 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-02-20T07:51:41.883 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:41 +0000 (0:00:00.163) 0:02:37.379 ***** 2021-02-20T07:51:42.006 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-02-20T07:51:42.007 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807501.7047298-138447866862299/source", "state": "file", "uid": 0} 2021-02-20T07:51:42.047 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:42 +0000 (0:00:00.163) 0:02:37.543 ***** 2021-02-20T07:51:42.238 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1613807501.9178047-63080228029204/source", "state": "file", "uid": 0} 2021-02-20T07:51:42.240 INFO:teuthology.task.ansible.out: 2021-02-20T07:51:42.281 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:42 +0000 (0:00:00.233) 0:02:37.777 ***** 2021-02-20T07:51:42.319 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-02-20T07:51:42.321 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.123825", "end": "2021-02-20 07:51:42.285582", "rc": 0, "start": "2021-02-20 07:51:42.161757", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:42.359 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:42 +0000 (0:00:00.078) 0:02:37.855 ***** 2021-02-20T07:51:42.559 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.147882", "end": "2021-02-20 07:51:42.516966", "rc": 0, "start": "2021-02-20 07:51:42.369084", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-02-20T07:51:42.597 INFO:teuthology.task.ansible.out:Saturday 20 February 2021 07:51:42 +0000 (0:00:00.238) 0:02:38.093 ***** 2021-02-20T07:51:45.552 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-02-20T07:51:45.553 INFO:teuthology.task.ansible.out:changed: [smithi114.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "docker.socket network-online.target systemd-journald.socket firewalld.service basic.target sysinit.target containerd.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T07:51:45.555 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-02-20T07:51:45.557 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target docker.socket", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:50:38 UTC", "StateChangeTimestampMonotonic": "125083169", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T07:51:45.559 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T07:51:46.076 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sysinit.target firewalld.service docker.socket systemd-journald.socket network-online.target basic.target system.slice containerd.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-02-20T07:51:46.078 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-02-20T07:51:46.079 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sat 2021-02-20 07:50:39 UTC", "StateChangeTimestampMonotonic": "132025537", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "S 2021-02-20T07:51:46.081 INFO:teuthology.task.ansible.out:uccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-02-20T07:51:46.186 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-02-20T07:51:46.191 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-02-20T07:51:46.196 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-02-20T07:51:46.197 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi106.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 smithi114.front.sepia.ceph.com : ok=126 changed=38 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Saturday 20 February 2021 07:51:46 +0000 (0:00:03.599) 0:02:41.692 ***** =============================================================================== users : Create all admin users with sudo access. ----------------------- 19.53s users : Update authorized_keys using the keys repo --------------------- 19.07s testnode : Zap all non-root disks --------------------------------------- 8.50s testnode : Install packages --------------------------------------------- 8.42s testnode : Ensure packages are not present. ----------------------------- 8.41s users : Remove revoked users -------------------------------------------- 6.43s container-host : Restart docker service --------------------------------- 3.60s container-host : Install registries-conf-ctl ---------------------------- 3.54s common : Update apt cache ----------------------------------------------- 2.91s testnode : Update apt cache. -------------------------------------------- 2.20s Gathering Facts --------------------------------------------------------- 1.85s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.79s testnode : Create logical volume(s) ------------------------------------- 1.51s testnode : Install apt keys --------------------------------------------- 1.32s users : Update authorized_keys for each user with literal keys ---------- 1.31s testnode : Install packages via pip ------------------------------------- 1.00s testnode : Upgrade packages --------------------------------------------- 0.94s testnode : Install ntp package on apt based systems. -------------------- 0.91s testnode : Ensure perl-doc is installed on apt 2021-02-20T07:51:46.199 INFO:teuthology.task.ansible.out: systems. ----------------- 0.90s testnode : Install python-pip on apt based systems. --------------------- 0.87s 2021-02-20T07:51:46.554 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2021-02-20T07:51:46.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:51:46.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:51:46.633 DEBUG:teuthology.orchestra.run.smithi106:> true 2021-02-20T07:51:46.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-20T07:51:46.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-20T07:51:46.913 DEBUG:teuthology.orchestra.run.smithi114:> true 2021-02-20T07:51:47.116 INFO:teuthology.run_tasks:Running task clock... 2021-02-20T07:51:47.128 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-02-20T07:51:47.128 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T07:51:47.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T07:51:47.182 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T07:51:47.182 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Command line: ntpd -gq 2021-02-20T07:51:47.183 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: proto: precision = 0.123 usec (-23) 2021-02-20T07:51:47.184 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T07:51:47.184 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T07:51:47.184 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T07:51:47.184 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T07:51:47.185 INFO:teuthology.orchestra.run.smithi106.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T07:51:47.185 INFO:teuthology.orchestra.run.smithi106.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T07:51:47.186 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T07:51:47.186 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen normally on 3 enp3s0f1 172.21.15.106:123 2021-02-20T07:51:47.186 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen normally on 4 lo [::1]:123 2021-02-20T07:51:47.187 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce6d%5]:123 2021-02-20T07:51:47.187 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:47 ntpd[8212]: Listening on routing socket on fd #22 for interface updates 2021-02-20T07:51:48.183 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:48 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:48.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:48 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:48.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:48 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:48.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:48 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:50.183 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:50.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:50.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:50.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:50.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:50.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:50.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:50.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:50 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:52.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:52.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:52.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:52.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:52.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:52.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:52.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:52.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:52.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:52.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:52.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:52.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:52.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:52.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:52.188 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:52.188 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:52 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:54.183 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:54.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:54.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:54.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:54.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:54.184 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:54.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:54.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:54.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:54.185 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:54.186 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:54.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:54.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:54.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:54.187 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:54.188 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:54.188 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T07:51:54.188 INFO:teuthology.orchestra.run.smithi106.stderr:20 Feb 07:51:54 ntpd[8212]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T07:51:54.188 INFO:teuthology.orchestra.run.smithi106.stdout:20 Feb 07:51:54 ntpd[8212]: ntpd: time slew +0.024798 s 2021-02-20T07:51:54.188 INFO:teuthology.orchestra.run.smithi106.stdout:ntpd: time slew +0.024798s 2021-02-20T07:51:54.231 INFO:teuthology.orchestra.run.smithi106.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T07:51:54.231 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================== 2021-02-20T07:51:54.232 INFO:teuthology.orchestra.run.smithi106.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:51:54.232 INFO:teuthology.orchestra.run.smithi106.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:51:54.232 INFO:teuthology.orchestra.run.smithi106.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:51:54.232 INFO:teuthology.orchestra.run.smithi106.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:51:54.233 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-02-20T07:51:54.233 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-02-20T07:51:54.311 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-02-20T07:51:54.312 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Command line: ntpd -gq 2021-02-20T07:51:54.313 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: proto: precision = 0.123 usec (-23) 2021-02-20T07:51:54.313 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T07:51:54.313 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: restrict ::: KOD does nothing without LIMITED. 2021-02-20T07:51:54.314 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen and drop on 0 v6wildcard [::]:123 2021-02-20T07:51:54.314 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-02-20T07:51:54.314 INFO:teuthology.orchestra.run.smithi114.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-02-20T07:51:54.314 INFO:teuthology.orchestra.run.smithi114.stderr:restrict ::: KOD does nothing without LIMITED. 2021-02-20T07:51:54.315 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen normally on 2 lo 127.0.0.1:123 2021-02-20T07:51:54.315 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen normally on 3 enp3s0f1 172.21.15.114:123 2021-02-20T07:51:54.316 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen normally on 4 lo [::1]:123 2021-02-20T07:51:54.316 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cb63%5]:123 2021-02-20T07:51:54.316 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:51:54 ntpd[8207]: Listening on routing socket on fd #22 for interface updates 2021-02-20T07:51:55.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:55 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:55.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:55 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:55.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:55 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:55.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:55 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:56.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:56.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:56.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:56.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:56.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:56.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:56.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:56.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:56 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:57.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:57 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:57.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:57 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:57.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:57 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:57.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:57 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:58.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:58.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:58.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:58.315 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:58.315 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:58.315 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:58.316 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:58.316 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:58 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:59.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:59.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:59.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:59.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:51:59.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:51:59.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:51:59.315 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:51:59.315 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:51:59 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:52:00.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:52:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:52:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:52:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:52:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:52:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:52:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:52:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:00 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:52:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-02-20T07:52:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: can't open /var/log/ntpstats/rawstats.20210220: Permission denied 2021-02-20T07:52:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-02-20T07:52:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: can't open /var/log/ntpstats/peerstats.20210220: Permission denied 2021-02-20T07:52:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-02-20T07:52:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:20 Feb 07:52:01 ntpd[8207]: can't open /var/log/ntpstats/loopstats.20210220: Permission denied 2021-02-20T07:52:01.314 INFO:teuthology.orchestra.run.smithi114.stdout:20 Feb 07:52:01 ntpd[8207]: ntpd: time slew +0.009063 s 2021-02-20T07:52:01.314 INFO:teuthology.orchestra.run.smithi114.stdout:ntpd: time slew +0.009063s 2021-02-20T07:52:01.363 INFO:teuthology.orchestra.run.smithi114.stdout: remote refid st t when poll reach delay offset jitter 2021-02-20T07:52:01.364 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================== 2021-02-20T07:52:01.364 INFO:teuthology.orchestra.run.smithi114.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:52:01.364 INFO:teuthology.orchestra.run.smithi114.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:52:01.365 INFO:teuthology.orchestra.run.smithi114.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:52:01.365 INFO:teuthology.orchestra.run.smithi114.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-02-20T07:52:01.366 INFO:teuthology.run_tasks:Running task install... 2021-02-20T07:52:01.377 DEBUG:teuthology.task.install:project ceph 2021-02-20T07:52:01.377 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}} 2021-02-20T07:52:01.378 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'} 2021-02-20T07:52:01.378 INFO:teuthology.task.install:Using flavor: basic 2021-02-20T07:52:01.385 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-02-20T07:52:01.386 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2021-02-20T07:52:01.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2021-02-20T07:52:01.440 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2021-02-20T07:52:01.445 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:52:01.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo lsb_release -is 2021-02-20T07:52:01.446 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-key list | grep Ceph 2021-02-20T07:52:01.499 INFO:teuthology.orchestra.run.smithi114.stdout:Ubuntu 2021-02-20T07:52:01.504 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:52:01.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-key list | grep Ceph 2021-02-20T07:52:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T07:52:01.592 INFO:teuthology.orchestra.run.smithi114.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-02-20T07:52:01.595 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T07:52:01.596 INFO:teuthology.orchestra.run.smithi106.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T07:52:01.597 DEBUG:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2021-02-20T07:52:01.645 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2021-02-20T07:52:01.648 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-02-20T07:52:01.649 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph.com (release key) 2021-02-20T07:52:01.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo lsb_release -is 2021-02-20T07:52:01.651 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:52:01.651 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T07:52:01.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:52:01.707 INFO:teuthology.orchestra.run.smithi114.stdout:Ubuntu 2021-02-20T07:52:01.712 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:52:01.712 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-02-20T07:52:01.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:52:01.832 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default/ 2021-02-20T07:52:01.833 INFO:teuthology.task.install.deb:Package version is 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:52:01.874 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default/ 2021-02-20T07:52:01.875 INFO:teuthology.task.install.deb:Package version is 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:52:01.960 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:52:01.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T07:52:01.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo apt-get update 2021-02-20T07:52:01.999 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:52:01.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-02-20T07:52:02.015 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2021-02-20T07:52:02.193 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:52:02.238 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T07:52:02.255 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-02-20T07:52:02.258 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic InRelease 2021-02-20T07:52:02.261 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T07:52:02.282 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-02-20T07:52:02.287 INFO:teuthology.orchestra.run.smithi106.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T07:52:02.307 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic Release.gpg 2021-02-20T07:52:02.310 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2021-02-20T07:52:02.330 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T07:52:02.336 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:52:02.345 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.6 kB] 2021-02-20T07:52:02.350 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-02-20T07:52:02.372 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2021-02-20T07:52:02.417 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:52:02.430 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-02-20T07:52:02.494 INFO:teuthology.orchestra.run.smithi106.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:52:02.510 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-02-20T07:52:02.817 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 27.4 kB in 1s (45.2 kB/s) 2021-02-20T07:52:02.857 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 27.4 kB in 1s (40.2 kB/s) 2021-02-20T07:52:04.049 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:52:04.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-929-gd318d42d-1bionic cephadm=17.0.0-929-gd318d42d-1bionic ceph-mds=17.0.0-929-gd318d42d-1bionic ceph-mgr=17.0.0-929-gd318d42d-1bionic ceph-common=17.0.0-929-gd318d42d-1bionic ceph-fuse=17.0.0-929-gd318d42d-1bionic ceph-test=17.0.0-929-gd318d42d-1bionic radosgw=17.0.0-929-gd318d42d-1bionic python3-rados=17.0.0-929-gd318d42d-1bionic python3-rgw=17.0.0-929-gd318d42d-1bionic python3-cephfs=17.0.0-929-gd318d42d-1bionic python3-rbd=17.0.0-929-gd318d42d-1bionic libcephfs2=17.0.0-929-gd318d42d-1bionic libcephfs-dev=17.0.0-929-gd318d42d-1bionic librados2=17.0.0-929-gd318d42d-1bionic librbd1=17.0.0-929-gd318d42d-1bionic rbd-fuse=17.0.0-929-gd318d42d-1bionic rbd-mirror=17.0.0-929-gd318d42d-1bionic 2021-02-20T07:52:04.111 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:52:04.158 DEBUG:teuthology.orchestra.run.smithi106:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-929-gd318d42d-1bionic cephadm=17.0.0-929-gd318d42d-1bionic ceph-mds=17.0.0-929-gd318d42d-1bionic ceph-mgr=17.0.0-929-gd318d42d-1bionic ceph-common=17.0.0-929-gd318d42d-1bionic ceph-fuse=17.0.0-929-gd318d42d-1bionic ceph-test=17.0.0-929-gd318d42d-1bionic radosgw=17.0.0-929-gd318d42d-1bionic python3-rados=17.0.0-929-gd318d42d-1bionic python3-rgw=17.0.0-929-gd318d42d-1bionic python3-cephfs=17.0.0-929-gd318d42d-1bionic python3-rbd=17.0.0-929-gd318d42d-1bionic libcephfs2=17.0.0-929-gd318d42d-1bionic libcephfs-dev=17.0.0-929-gd318d42d-1bionic librados2=17.0.0-929-gd318d42d-1bionic librbd1=17.0.0-929-gd318d42d-1bionic rbd-fuse=17.0.0-929-gd318d42d-1bionic rbd-mirror=17.0.0-929-gd318d42d-1bionic 2021-02-20T07:52:04.172 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2021-02-20T07:52:04.235 INFO:teuthology.orchestra.run.smithi106.stdout:Reading package lists... 2021-02-20T07:52:04.430 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2021-02-20T07:52:04.430 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2021-02-20T07:52:04.466 INFO:teuthology.orchestra.run.smithi106.stdout:Building dependency tree... 2021-02-20T07:52:04.467 INFO:teuthology.orchestra.run.smithi106.stdout:Reading state information... 2021-02-20T07:52:04.597 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2021-02-20T07:52:04.598 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T07:52:04.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T07:52:04.599 INFO:teuthology.orchestra.run.smithi114.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T07:52:04.600 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T07:52:04.600 INFO:teuthology.orchestra.run.smithi114.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T07:52:04.600 INFO:teuthology.orchestra.run.smithi114.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T07:52:04.600 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T07:52:04.601 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T07:52:04.601 INFO:teuthology.orchestra.run.smithi114.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T07:52:04.601 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T07:52:04.601 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T07:52:04.602 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T07:52:04.602 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T07:52:04.602 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T07:52:04.602 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T07:52:04.602 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T07:52:04.603 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T07:52:04.603 INFO:teuthology.orchestra.run.smithi114.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T07:52:04.603 INFO:teuthology.orchestra.run.smithi114.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T07:52:04.603 INFO:teuthology.orchestra.run.smithi114.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T07:52:04.603 INFO:teuthology.orchestra.run.smithi114.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T07:52:04.604 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T07:52:04.604 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2021-02-20T07:52:04.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T07:52:04.605 INFO:teuthology.orchestra.run.smithi114.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T07:52:04.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T07:52:04.605 INFO:teuthology.orchestra.run.smithi114.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T07:52:04.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T07:52:04.606 INFO:teuthology.orchestra.run.smithi114.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T07:52:04.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T07:52:04.606 INFO:teuthology.orchestra.run.smithi114.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T07:52:04.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T07:52:04.607 INFO:teuthology.orchestra.run.smithi114.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T07:52:04.607 INFO:teuthology.orchestra.run.smithi114.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T07:52:04.607 INFO:teuthology.orchestra.run.smithi114.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T07:52:04.607 INFO:teuthology.orchestra.run.smithi114.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T07:52:04.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T07:52:04.608 INFO:teuthology.orchestra.run.smithi114.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T07:52:04.634 INFO:teuthology.orchestra.run.smithi106.stdout:The following additional packages will be installed: 2021-02-20T07:52:04.634 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-02-20T07:52:04.635 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-02-20T07:52:04.635 INFO:teuthology.orchestra.run.smithi106.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-02-20T07:52:04.636 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-02-20T07:52:04.636 INFO:teuthology.orchestra.run.smithi106.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-02-20T07:52:04.636 INFO:teuthology.orchestra.run.smithi106.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T07:52:04.637 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-02-20T07:52:04.637 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-02-20T07:52:04.637 INFO:teuthology.orchestra.run.smithi106.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-02-20T07:52:04.637 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-02-20T07:52:04.637 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-02-20T07:52:04.638 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-02-20T07:52:04.638 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-02-20T07:52:04.638 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-02-20T07:52:04.638 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-02-20T07:52:04.638 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-02-20T07:52:04.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-02-20T07:52:04.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-02-20T07:52:04.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-02-20T07:52:04.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-02-20T07:52:04.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-02-20T07:52:04.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-02-20T07:52:04.640 INFO:teuthology.orchestra.run.smithi106.stdout:Suggested packages: 2021-02-20T07:52:04.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-02-20T07:52:04.641 INFO:teuthology.orchestra.run.smithi106.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-02-20T07:52:04.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-02-20T07:52:04.641 INFO:teuthology.orchestra.run.smithi106.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-02-20T07:52:04.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-02-20T07:52:04.642 INFO:teuthology.orchestra.run.smithi106.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-02-20T07:52:04.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-02-20T07:52:04.642 INFO:teuthology.orchestra.run.smithi106.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-02-20T07:52:04.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-02-20T07:52:04.643 INFO:teuthology.orchestra.run.smithi106.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-02-20T07:52:04.643 INFO:teuthology.orchestra.run.smithi106.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-02-20T07:52:04.643 INFO:teuthology.orchestra.run.smithi106.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-02-20T07:52:04.643 INFO:teuthology.orchestra.run.smithi106.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-02-20T07:52:04.644 INFO:teuthology.orchestra.run.smithi106.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-02-20T07:52:04.644 INFO:teuthology.orchestra.run.smithi106.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-02-20T07:52:04.703 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2021-02-20T07:52:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T07:52:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T07:52:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T07:52:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T07:52:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T07:52:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T07:52:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T07:52:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T07:52:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T07:52:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T07:52:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T07:52:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T07:52:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T07:52:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T07:52:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T07:52:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T07:52:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T07:52:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T07:52:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T07:52:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T07:52:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T07:52:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T07:52:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T07:52:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T07:52:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T07:52:04.711 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages will be upgraded: 2021-02-20T07:52:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 librbd1 2021-02-20T07:52:04.743 INFO:teuthology.orchestra.run.smithi106.stdout:The following NEW packages will be installed: 2021-02-20T07:52:04.743 INFO:teuthology.orchestra.run.smithi106.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-02-20T07:52:04.743 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-02-20T07:52:04.744 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-02-20T07:52:04.745 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-02-20T07:52:04.745 INFO:teuthology.orchestra.run.smithi106.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-02-20T07:52:04.745 INFO:teuthology.orchestra.run.smithi106.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-02-20T07:52:04.746 INFO:teuthology.orchestra.run.smithi106.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-02-20T07:52:04.746 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-02-20T07:52:04.746 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-02-20T07:52:04.746 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-02-20T07:52:04.747 INFO:teuthology.orchestra.run.smithi106.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-02-20T07:52:04.747 INFO:teuthology.orchestra.run.smithi106.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-02-20T07:52:04.747 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-02-20T07:52:04.747 INFO:teuthology.orchestra.run.smithi106.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-02-20T07:52:04.747 INFO:teuthology.orchestra.run.smithi106.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-02-20T07:52:04.748 INFO:teuthology.orchestra.run.smithi106.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-02-20T07:52:04.748 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-02-20T07:52:04.748 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-02-20T07:52:04.748 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-02-20T07:52:04.749 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-02-20T07:52:04.749 INFO:teuthology.orchestra.run.smithi106.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-02-20T07:52:04.749 INFO:teuthology.orchestra.run.smithi106.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-02-20T07:52:04.749 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-02-20T07:52:04.749 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-02-20T07:52:04.750 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-02-20T07:52:04.750 INFO:teuthology.orchestra.run.smithi106.stdout:The following packages will be upgraded: 2021-02-20T07:52:04.751 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 librbd1 2021-02-20T07:52:04.841 INFO:teuthology.orchestra.run.smithi114.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T07:52:04.842 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 169 MB of archives. 2021-02-20T07:52:04.842 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T07:52:04.842 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-929-gd318d42d-1bionic [3,712 B] 2021-02-20T07:52:04.885 INFO:teuthology.orchestra.run.smithi106.stdout:2 upgraded, 112 newly installed, 0 to remove and 34 not upgraded. 2021-02-20T07:52:04.885 INFO:teuthology.orchestra.run.smithi106.stdout:Need to get 169 MB of archives. 2021-02-20T07:52:04.885 INFO:teuthology.orchestra.run.smithi106.stdout:After this operation, 793 MB of additional disk space will be used. 2021-02-20T07:52:04.886 INFO:teuthology.orchestra.run.smithi106.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 17.0.0-929-gd318d42d-1bionic [3,712 B] 2021-02-20T07:52:04.888 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-929-gd318d42d-1bionic [3,122 kB] 2021-02-20T07:52:04.892 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T07:52:04.931 INFO:teuthology.orchestra.run.smithi106.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 17.0.0-929-gd318d42d-1bionic [3,122 kB] 2021-02-20T07:52:04.933 INFO:teuthology.orchestra.run.smithi106.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-02-20T07:52:04.973 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T07:52:05.012 INFO:teuthology.orchestra.run.smithi106.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-02-20T07:52:05.145 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-929-gd318d42d-1bionic [3,162 kB] 2021-02-20T07:52:05.190 INFO:teuthology.orchestra.run.smithi106.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 17.0.0-929-gd318d42d-1bionic [3,162 kB] 2021-02-20T07:52:05.232 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T07:52:05.236 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T07:52:05.236 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T07:52:05.263 INFO:teuthology.orchestra.run.smithi106.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-02-20T07:52:05.264 INFO:teuthology.orchestra.run.smithi106.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-02-20T07:52:05.266 INFO:teuthology.orchestra.run.smithi106.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-02-20T07:52:05.345 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-929-gd318d42d-1bionic [664 kB] 2021-02-20T07:52:05.352 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T07:52:05.355 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T07:52:05.380 INFO:teuthology.orchestra.run.smithi106.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-02-20T07:52:05.383 INFO:teuthology.orchestra.run.smithi106.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-02-20T07:52:05.388 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-929-gd318d42d-1bionic [339 kB] 2021-02-20T07:52:05.390 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T07:52:05.397 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T07:52:05.401 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-929-gd318d42d-1bionic [21.7 kB] 2021-02-20T07:52:05.402 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-929-gd318d42d-1bionic [162 kB] 2021-02-20T07:52:05.406 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T07:52:05.408 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-929-gd318d42d-1bionic [29.2 kB] 2021-02-20T07:52:05.415 INFO:teuthology.orchestra.run.smithi106.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 17.0.0-929-gd318d42d-1bionic [664 kB] 2021-02-20T07:52:05.418 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T07:52:05.421 INFO:teuthology.orchestra.run.smithi106.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-02-20T07:52:05.424 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-929-gd318d42d-1bionic [336 kB] 2021-02-20T07:52:05.428 INFO:teuthology.orchestra.run.smithi106.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-02-20T07:52:05.428 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T07:52:05.437 INFO:teuthology.orchestra.run.smithi106.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-02-20T07:52:05.444 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-929-gd318d42d-1bionic [3,295 kB] 2021-02-20T07:52:05.450 INFO:teuthology.orchestra.run.smithi106.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-02-20T07:52:05.456 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T07:52:05.459 INFO:teuthology.orchestra.run.smithi106.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-02-20T07:52:05.460 INFO:teuthology.orchestra.run.smithi114.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T07:52:05.461 INFO:teuthology.orchestra.run.smithi106.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 17.0.0-929-gd318d42d-1bionic [339 kB] 2021-02-20T07:52:05.473 INFO:teuthology.orchestra.run.smithi114.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T07:52:05.474 INFO:teuthology.orchestra.run.smithi106.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 17.0.0-929-gd318d42d-1bionic [21.7 kB] 2021-02-20T07:52:05.474 INFO:teuthology.orchestra.run.smithi106.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 17.0.0-929-gd318d42d-1bionic [162 kB] 2021-02-20T07:52:05.482 INFO:teuthology.orchestra.run.smithi106.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-02-20T07:52:05.483 INFO:teuthology.orchestra.run.smithi106.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 17.0.0-929-gd318d42d-1bionic [29.2 kB] 2021-02-20T07:52:05.484 INFO:teuthology.orchestra.run.smithi114.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T07:52:05.484 INFO:teuthology.orchestra.run.smithi114.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T07:52:05.486 INFO:teuthology.orchestra.run.smithi106.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 17.0.0-929-gd318d42d-1bionic [336 kB] 2021-02-20T07:52:05.486 INFO:teuthology.orchestra.run.smithi106.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-02-20T07:52:05.501 INFO:teuthology.orchestra.run.smithi106.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-02-20T07:52:05.511 INFO:teuthology.orchestra.run.smithi106.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-02-20T07:52:05.513 INFO:teuthology.orchestra.run.smithi106.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-02-20T07:52:05.517 INFO:teuthology.orchestra.run.smithi106.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 17.0.0-929-gd318d42d-1bionic [3,295 kB] 2021-02-20T07:52:05.552 INFO:teuthology.orchestra.run.smithi114.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T07:52:05.553 INFO:teuthology.orchestra.run.smithi114.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T07:52:05.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T07:52:05.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T07:52:05.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T07:52:05.559 INFO:teuthology.orchestra.run.smithi114.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T07:52:05.561 INFO:teuthology.orchestra.run.smithi114.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T07:52:05.579 INFO:teuthology.orchestra.run.smithi106.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-02-20T07:52:05.580 INFO:teuthology.orchestra.run.smithi106.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-02-20T07:52:05.582 INFO:teuthology.orchestra.run.smithi106.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-02-20T07:52:05.583 INFO:teuthology.orchestra.run.smithi106.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-02-20T07:52:05.583 INFO:teuthology.orchestra.run.smithi106.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-02-20T07:52:05.586 INFO:teuthology.orchestra.run.smithi106.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-02-20T07:52:05.588 INFO:teuthology.orchestra.run.smithi106.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-02-20T07:52:05.631 INFO:teuthology.orchestra.run.smithi114.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T07:52:05.632 INFO:teuthology.orchestra.run.smithi114.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T07:52:05.652 INFO:teuthology.orchestra.run.smithi114.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T07:52:05.653 INFO:teuthology.orchestra.run.smithi114.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-929-gd318d42d-1bionic [99.2 kB] 2021-02-20T07:52:05.657 INFO:teuthology.orchestra.run.smithi106.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-02-20T07:52:05.658 INFO:teuthology.orchestra.run.smithi114.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-929-gd318d42d-1bionic [414 kB] 2021-02-20T07:52:05.659 INFO:teuthology.orchestra.run.smithi106.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-02-20T07:52:05.671 INFO:teuthology.orchestra.run.smithi114.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-929-gd318d42d-1bionic [21.2 MB] 2021-02-20T07:52:05.678 INFO:teuthology.orchestra.run.smithi106.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-02-20T07:52:05.711 INFO:teuthology.orchestra.run.smithi114.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T07:52:05.711 INFO:teuthology.orchestra.run.smithi114.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T07:52:05.712 INFO:teuthology.orchestra.run.smithi114.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T07:52:05.716 INFO:teuthology.orchestra.run.smithi114.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T07:52:05.724 INFO:teuthology.orchestra.run.smithi114.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T07:52:05.727 INFO:teuthology.orchestra.run.smithi114.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T07:52:05.728 INFO:teuthology.orchestra.run.smithi114.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T07:52:05.736 INFO:teuthology.orchestra.run.smithi106.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-02-20T07:52:05.737 INFO:teuthology.orchestra.run.smithi106.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-02-20T07:52:05.739 INFO:teuthology.orchestra.run.smithi106.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-02-20T07:52:05.744 INFO:teuthology.orchestra.run.smithi106.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-02-20T07:52:05.748 INFO:teuthology.orchestra.run.smithi106.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 17.0.0-929-gd318d42d-1bionic [99.2 kB] 2021-02-20T07:52:05.750 INFO:teuthology.orchestra.run.smithi106.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 17.0.0-929-gd318d42d-1bionic [414 kB] 2021-02-20T07:52:05.751 INFO:teuthology.orchestra.run.smithi106.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-02-20T07:52:05.753 INFO:teuthology.orchestra.run.smithi106.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-02-20T07:52:05.754 INFO:teuthology.orchestra.run.smithi106.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-02-20T07:52:05.766 INFO:teuthology.orchestra.run.smithi106.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 17.0.0-929-gd318d42d-1bionic [21.2 MB] 2021-02-20T07:52:05.790 INFO:teuthology.orchestra.run.smithi114.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T07:52:05.814 INFO:teuthology.orchestra.run.smithi106.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-02-20T07:52:05.877 INFO:teuthology.orchestra.run.smithi114.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T07:52:05.878 INFO:teuthology.orchestra.run.smithi114.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T07:52:05.899 INFO:teuthology.orchestra.run.smithi106.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-02-20T07:52:05.899 INFO:teuthology.orchestra.run.smithi106.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-02-20T07:52:06.151 INFO:teuthology.orchestra.run.smithi114.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T07:52:06.169 INFO:teuthology.orchestra.run.smithi106.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-02-20T07:52:06.195 INFO:teuthology.orchestra.run.smithi114.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T07:52:06.196 INFO:teuthology.orchestra.run.smithi114.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T07:52:06.210 INFO:teuthology.orchestra.run.smithi106.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-02-20T07:52:06.212 INFO:teuthology.orchestra.run.smithi106.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-02-20T07:52:06.232 INFO:teuthology.orchestra.run.smithi114.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T07:52:06.232 INFO:teuthology.orchestra.run.smithi114.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T07:52:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T07:52:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T07:52:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T07:52:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T07:52:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T07:52:06.246 INFO:teuthology.orchestra.run.smithi106.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-02-20T07:52:06.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-02-20T07:52:06.247 INFO:teuthology.orchestra.run.smithi106.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2021-02-20T07:52:06.248 INFO:teuthology.orchestra.run.smithi106.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-02-20T07:52:06.248 INFO:teuthology.orchestra.run.smithi106.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-02-20T07:52:06.249 INFO:teuthology.orchestra.run.smithi106.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-02-20T07:52:06.249 INFO:teuthology.orchestra.run.smithi106.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-02-20T07:52:06.310 INFO:teuthology.orchestra.run.smithi114.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T07:52:06.312 INFO:teuthology.orchestra.run.smithi114.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T07:52:06.326 INFO:teuthology.orchestra.run.smithi106.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-02-20T07:52:06.327 INFO:teuthology.orchestra.run.smithi106.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-02-20T07:52:06.333 INFO:teuthology.orchestra.run.smithi106.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T07:52:06.334 INFO:teuthology.orchestra.run.smithi106.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T07:52:06.388 INFO:teuthology.orchestra.run.smithi114.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-02-20T07:52:06.389 INFO:teuthology.orchestra.run.smithi114.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-02-20T07:52:06.393 INFO:teuthology.orchestra.run.smithi114.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T07:52:06.393 INFO:teuthology.orchestra.run.smithi114.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T07:52:06.394 INFO:teuthology.orchestra.run.smithi114.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T07:52:06.396 INFO:teuthology.orchestra.run.smithi114.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T07:52:06.399 INFO:teuthology.orchestra.run.smithi114.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T07:52:06.400 INFO:teuthology.orchestra.run.smithi114.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T07:52:06.404 INFO:teuthology.orchestra.run.smithi106.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-02-20T07:52:06.404 INFO:teuthology.orchestra.run.smithi106.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-02-20T07:52:06.405 INFO:teuthology.orchestra.run.smithi106.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-02-20T07:52:06.412 INFO:teuthology.orchestra.run.smithi106.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-02-20T07:52:06.412 INFO:teuthology.orchestra.run.smithi106.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-02-20T07:52:06.413 INFO:teuthology.orchestra.run.smithi106.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-02-20T07:52:06.468 INFO:teuthology.orchestra.run.smithi114.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T07:52:06.472 INFO:teuthology.orchestra.run.smithi114.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T07:52:06.482 INFO:teuthology.orchestra.run.smithi106.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-02-20T07:52:06.487 INFO:teuthology.orchestra.run.smithi106.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-02-20T07:52:06.494 INFO:teuthology.orchestra.run.smithi106.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T07:52:06.497 INFO:teuthology.orchestra.run.smithi106.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T07:52:06.547 INFO:teuthology.orchestra.run.smithi114.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-02-20T07:52:06.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-02-20T07:52:06.555 INFO:teuthology.orchestra.run.smithi114.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T07:52:06.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T07:52:06.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T07:52:06.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T07:52:06.558 INFO:teuthology.orchestra.run.smithi114.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T07:52:06.559 INFO:teuthology.orchestra.run.smithi114.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T07:52:06.560 INFO:teuthology.orchestra.run.smithi106.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-02-20T07:52:06.561 INFO:teuthology.orchestra.run.smithi106.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-02-20T07:52:06.562 INFO:teuthology.orchestra.run.smithi106.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-02-20T07:52:06.564 INFO:teuthology.orchestra.run.smithi106.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-02-20T07:52:06.566 INFO:teuthology.orchestra.run.smithi106.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-02-20T07:52:06.567 INFO:teuthology.orchestra.run.smithi106.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-02-20T07:52:06.627 INFO:teuthology.orchestra.run.smithi114.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T07:52:06.628 INFO:teuthology.orchestra.run.smithi114.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T07:52:06.639 INFO:teuthology.orchestra.run.smithi106.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-02-20T07:52:06.640 INFO:teuthology.orchestra.run.smithi106.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-02-20T07:52:06.643 INFO:teuthology.orchestra.run.smithi106.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T07:52:06.664 INFO:teuthology.orchestra.run.smithi106.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T07:52:06.706 INFO:teuthology.orchestra.run.smithi114.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-02-20T07:52:06.717 INFO:teuthology.orchestra.run.smithi106.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T07:52:06.718 INFO:teuthology.orchestra.run.smithi106.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T07:52:06.729 INFO:teuthology.orchestra.run.smithi114.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-02-20T07:52:06.729 INFO:teuthology.orchestra.run.smithi114.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-02-20T07:52:06.730 INFO:teuthology.orchestra.run.smithi114.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-02-20T07:52:06.818 INFO:teuthology.orchestra.run.smithi106.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T07:52:06.819 INFO:teuthology.orchestra.run.smithi106.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T07:52:06.819 INFO:teuthology.orchestra.run.smithi106.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T07:52:06.823 INFO:teuthology.orchestra.run.smithi106.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T07:52:06.823 INFO:teuthology.orchestra.run.smithi106.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T07:52:06.824 INFO:teuthology.orchestra.run.smithi106.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T07:52:06.825 INFO:teuthology.orchestra.run.smithi106.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T07:52:06.826 INFO:teuthology.orchestra.run.smithi106.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T07:52:06.868 INFO:teuthology.orchestra.run.smithi114.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-02-20T07:52:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-02-20T07:52:06.870 INFO:teuthology.orchestra.run.smithi114.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-02-20T07:52:06.873 INFO:teuthology.orchestra.run.smithi106.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T07:52:06.877 INFO:teuthology.orchestra.run.smithi114.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-02-20T07:52:06.877 INFO:teuthology.orchestra.run.smithi114.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-02-20T07:52:06.878 INFO:teuthology.orchestra.run.smithi114.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-02-20T07:52:06.879 INFO:teuthology.orchestra.run.smithi114.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-02-20T07:52:06.881 INFO:teuthology.orchestra.run.smithi114.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-02-20T07:52:06.882 INFO:teuthology.orchestra.run.smithi114.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-02-20T07:52:07.068 INFO:teuthology.orchestra.run.smithi114.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-929-gd318d42d-1bionic [5,630 kB] 2021-02-20T07:52:07.280 INFO:teuthology.orchestra.run.smithi106.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 17.0.0-929-gd318d42d-1bionic [5,630 kB] 2021-02-20T07:52:07.483 INFO:teuthology.orchestra.run.smithi114.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-929-gd318d42d-1bionic [185 kB] 2021-02-20T07:52:07.494 INFO:teuthology.orchestra.run.smithi114.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-929-gd318d42d-1bionic [1,366 kB] 2021-02-20T07:52:07.675 INFO:teuthology.orchestra.run.smithi114.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-929-gd318d42d-1bionic [6,660 kB] 2021-02-20T07:52:07.737 INFO:teuthology.orchestra.run.smithi106.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 17.0.0-929-gd318d42d-1bionic [185 kB] 2021-02-20T07:52:07.743 INFO:teuthology.orchestra.run.smithi106.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 17.0.0-929-gd318d42d-1bionic [1,366 kB] 2021-02-20T07:52:07.858 INFO:teuthology.orchestra.run.smithi106.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 17.0.0-929-gd318d42d-1bionic [6,660 kB] 2021-02-20T07:52:08.132 INFO:teuthology.orchestra.run.smithi114.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-929-gd318d42d-1bionic [24.9 MB] 2021-02-20T07:52:08.449 INFO:teuthology.orchestra.run.smithi106.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 17.0.0-929-gd318d42d-1bionic [24.9 MB] 2021-02-20T07:52:10.470 INFO:teuthology.orchestra.run.smithi114.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-929-gd318d42d-1bionic [3,804 B] 2021-02-20T07:52:10.471 INFO:teuthology.orchestra.run.smithi114.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-929-gd318d42d-1bionic [770 kB] 2021-02-20T07:52:10.556 INFO:teuthology.orchestra.run.smithi114.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-929-gd318d42d-1bionic [2,154 kB] 2021-02-20T07:52:10.788 INFO:teuthology.orchestra.run.smithi114.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-929-gd318d42d-1bionic [62.2 kB] 2021-02-20T07:52:10.792 INFO:teuthology.orchestra.run.smithi114.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-929-gd318d42d-1bionic [67.9 kB] 2021-02-20T07:52:10.795 INFO:teuthology.orchestra.run.smithi114.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-929-gd318d42d-1bionic [2,328 kB] 2021-02-20T07:52:11.000 INFO:teuthology.orchestra.run.smithi106.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 17.0.0-929-gd318d42d-1bionic [3,804 B] 2021-02-20T07:52:11.000 INFO:teuthology.orchestra.run.smithi106.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 17.0.0-929-gd318d42d-1bionic [770 kB] 2021-02-20T07:52:11.031 INFO:teuthology.orchestra.run.smithi114.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-929-gd318d42d-1bionic [7,884 kB] 2021-02-20T07:52:11.083 INFO:teuthology.orchestra.run.smithi106.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 17.0.0-929-gd318d42d-1bionic [2,154 kB] 2021-02-20T07:52:11.305 INFO:teuthology.orchestra.run.smithi106.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 17.0.0-929-gd318d42d-1bionic [62.2 kB] 2021-02-20T07:52:11.336 INFO:teuthology.orchestra.run.smithi106.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 17.0.0-929-gd318d42d-1bionic [67.9 kB] 2021-02-20T07:52:11.338 INFO:teuthology.orchestra.run.smithi106.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 17.0.0-929-gd318d42d-1bionic [2,328 kB] 2021-02-20T07:52:11.580 INFO:teuthology.orchestra.run.smithi106.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 17.0.0-929-gd318d42d-1bionic [7,884 kB] 2021-02-20T07:52:11.852 INFO:teuthology.orchestra.run.smithi114.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-929-gd318d42d-1bionic [3,924 B] 2021-02-20T07:52:11.853 INFO:teuthology.orchestra.run.smithi114.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-929-gd318d42d-1bionic [23.3 kB] 2021-02-20T07:52:11.853 INFO:teuthology.orchestra.run.smithi114.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-929-gd318d42d-1bionic [45.0 MB] 2021-02-20T07:52:12.451 INFO:teuthology.orchestra.run.smithi106.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 17.0.0-929-gd318d42d-1bionic [3,924 B] 2021-02-20T07:52:12.452 INFO:teuthology.orchestra.run.smithi106.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 17.0.0-929-gd318d42d-1bionic [23.3 kB] 2021-02-20T07:52:12.453 INFO:teuthology.orchestra.run.smithi106.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 17.0.0-929-gd318d42d-1bionic [45.0 MB] 2021-02-20T07:52:16.560 INFO:teuthology.orchestra.run.smithi114.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-929-gd318d42d-1bionic [19.8 kB] 2021-02-20T07:52:16.560 INFO:teuthology.orchestra.run.smithi114.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-929-gd318d42d-1bionic [10.1 MB] 2021-02-20T07:52:17.476 INFO:teuthology.orchestra.run.smithi106.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 17.0.0-929-gd318d42d-1bionic [19.8 kB] 2021-02-20T07:52:17.477 INFO:teuthology.orchestra.run.smithi106.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 17.0.0-929-gd318d42d-1bionic [10.1 MB] 2021-02-20T07:52:17.639 INFO:teuthology.orchestra.run.smithi114.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-929-gd318d42d-1bionic [77.6 kB] 2021-02-20T07:52:17.641 INFO:teuthology.orchestra.run.smithi114.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-929-gd318d42d-1bionic [3,331 kB] 2021-02-20T07:52:18.423 INFO:teuthology.orchestra.run.smithi106.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 17.0.0-929-gd318d42d-1bionic [77.6 kB] 2021-02-20T07:52:18.430 INFO:teuthology.orchestra.run.smithi106.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-mgolub-testing/d318d42d502ea13c78f2af2499efd19dcc7e174b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 17.0.0-929-gd318d42d-1bionic [3,331 kB] 2021-02-20T07:52:18.699 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 169 MB in 13s (12.8 MB/s) 2021-02-20T07:52:18.805 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T07:52:18.853 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T07:52:18.856 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T07:52:18.895 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T07:52:19.148 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libjaeger. 2021-02-20T07:52:19.164 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../001-libjaeger_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:19.206 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:19.401 INFO:teuthology.orchestra.run.smithi106.stdout:Fetched 169 MB in 14s (12.2 MB/s) 2021-02-20T07:52:19.512 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-uritemplate. 2021-02-20T07:52:19.550 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T07:52:19.560 INFO:teuthology.orchestra.run.smithi106.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137611 files and directories currently installed.) 2021-02-20T07:52:19.564 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-02-20T07:52:19.564 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T07:52:19.602 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T07:52:19.642 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T07:52:19.805 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libjaeger. 2021-02-20T07:52:19.818 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../001-libjaeger_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:19.847 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:19.984 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../003-librbd1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:20.105 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librbd1 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T07:52:20.149 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-02-20T07:52:20.163 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-02-20T07:52:20.250 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T07:52:20.584 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../003-librbd1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:20.737 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librbd1 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T07:52:20.748 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../004-librados2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:20.876 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librados2 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T07:52:21.361 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../004-librados2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:21.457 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librados2 (17.0.0-929-gd318d42d-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-02-20T07:52:21.529 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T07:52:21.542 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:21.579 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:21.882 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rados. 2021-02-20T07:52:21.895 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../006-python3-rados_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:21.932 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.055 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs2. 2021-02-20T07:52:22.071 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../005-libcephfs2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:22.105 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.285 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T07:52:22.299 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:22.343 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.365 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rados. 2021-02-20T07:52:22.378 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../006-python3-rados_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:22.407 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.595 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T07:52:22.608 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:22.645 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.743 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-02-20T07:52:22.756 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../007-python3-ceph-argparse_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:22.793 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:22.956 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T07:52:22.972 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:23.015 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:23.045 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cephfs. 2021-02-20T07:52:23.059 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../008-python3-cephfs_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:23.095 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:23.317 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T07:52:23.333 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T07:52:23.381 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-ceph-common. 2021-02-20T07:52:23.392 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T07:52:23.394 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../009-python3-ceph-common_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:23.423 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:23.651 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-prettytable. 2021-02-20T07:52:23.662 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T07:52:23.666 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-02-20T07:52:23.675 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:23.700 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-02-20T07:52:23.720 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:23.936 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rbd. 2021-02-20T07:52:23.950 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../011-python3-rbd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:23.995 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:24.081 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T07:52:24.095 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T07:52:24.140 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T07:52:24.439 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-02-20T07:52:24.455 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-02-20T07:52:24.467 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T07:52:24.481 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T07:52:24.514 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T07:52:24.517 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T07:52:24.767 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-02-20T07:52:24.781 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-02-20T07:52:24.817 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T07:52:24.854 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librgw2. 2021-02-20T07:52:24.867 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../014-librgw2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:24.912 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:25.086 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package librgw2. 2021-02-20T07:52:25.102 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../014-librgw2_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:25.136 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:25.498 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T07:52:25.511 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:25.548 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:25.697 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rgw. 2021-02-20T07:52:25.710 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../015-python3-rgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:25.747 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:25.884 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T07:52:25.897 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T07:52:25.934 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T07:52:26.066 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-02-20T07:52:26.080 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-02-20T07:52:26.124 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T07:52:26.245 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T07:52:26.259 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T07:52:26.303 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T07:52:26.389 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-02-20T07:52:26.405 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-02-20T07:52:26.444 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T07:52:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liboath0. 2021-02-20T07:52:26.672 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T07:52:26.714 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T07:52:26.822 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package liboath0. 2021-02-20T07:52:26.836 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-02-20T07:52:26.880 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-02-20T07:52:27.034 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T07:52:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:27.092 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:27.191 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libradosstriper1. 2021-02-20T07:52:27.204 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../019-libradosstriper1_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:27.249 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:27.562 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-common. 2021-02-20T07:52:27.577 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../020-ceph-common_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:27.610 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-common. 2021-02-20T07:52:27.620 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:27.626 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../020-ceph-common_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:27.660 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:29.559 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-base. 2021-02-20T07:52:29.572 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../021-ceph-base_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:29.590 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-base. 2021-02-20T07:52:29.604 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../021-ceph-base_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:29.618 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:29.649 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:30.514 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T07:52:30.527 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T07:52:30.554 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-dateutil. 2021-02-20T07:52:30.569 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-02-20T07:52:30.572 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T07:52:30.604 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-02-20T07:52:30.892 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T07:52:30.906 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-asn1crypto. 2021-02-20T07:52:30.907 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T07:52:30.921 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-02-20T07:52:30.950 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T07:52:30.965 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-02-20T07:52:31.234 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T07:52:31.250 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T07:52:31.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cffi-backend. 2021-02-20T07:52:31.267 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-02-20T07:52:31.292 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T07:52:31.311 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-02-20T07:52:31.620 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T07:52:31.634 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T07:52:31.647 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cryptography. 2021-02-20T07:52:31.662 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-02-20T07:52:31.678 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T07:52:31.705 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T07:52:32.057 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T07:52:32.067 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-openssl. 2021-02-20T07:52:32.070 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T07:52:32.082 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-02-20T07:52:32.107 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T07:52:32.125 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T07:52:32.378 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T07:52:32.392 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:32.409 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-02-20T07:52:32.425 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:32.428 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:32.467 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:32.783 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T07:52:32.796 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T07:52:32.841 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T07:52:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-bcrypt. 2021-02-20T07:52:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-02-20T07:52:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-02-20T07:52:33.177 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T07:52:33.191 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cherrypy3. 2021-02-20T07:52:33.193 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T07:52:33.207 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-02-20T07:52:33.235 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T07:52:33.249 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-02-20T07:52:33.594 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T07:52:33.608 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T07:52:33.639 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jwt. 2021-02-20T07:52:33.652 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-02-20T07:52:33.653 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T07:52:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-02-20T07:52:33.997 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T07:52:34.000 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-logutils. 2021-02-20T07:52:34.012 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T07:52:34.015 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-02-20T07:52:34.055 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T07:52:34.058 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-02-20T07:52:34.353 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T07:52:34.369 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T07:52:34.375 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-markupsafe. 2021-02-20T07:52:34.388 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-02-20T07:52:34.411 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T07:52:34.433 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-02-20T07:52:34.706 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-mako. 2021-02-20T07:52:34.719 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-mako. 2021-02-20T07:52:34.720 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T07:52:34.735 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-02-20T07:52:34.764 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T07:52:34.777 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-02-20T07:52:35.083 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T07:52:35.089 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplegeneric. 2021-02-20T07:52:35.098 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T07:52:35.104 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-02-20T07:52:35.147 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T07:52:35.150 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-02-20T07:52:35.454 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T07:52:35.468 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T07:52:35.470 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-singledispatch. 2021-02-20T07:52:35.484 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-02-20T07:52:35.513 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T07:52:35.553 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-02-20T07:52:35.800 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webob. 2021-02-20T07:52:35.816 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T07:52:35.856 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webob. 2021-02-20T07:52:35.866 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T07:52:35.871 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-02-20T07:52:35.914 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T07:52:36.228 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T07:52:36.242 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T07:52:36.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-bs4. 2021-02-20T07:52:36.268 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-02-20T07:52:36.278 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T07:52:36.320 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-02-20T07:52:36.606 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T07:52:36.620 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T07:52:36.665 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T07:52:37.347 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-waitress. 2021-02-20T07:52:37.363 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-02-20T07:52:37.406 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-02-20T07:52:37.701 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T07:52:37.715 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T07:52:37.725 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tempita. 2021-02-20T07:52:37.739 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-02-20T07:52:37.759 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T07:52:37.775 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-02-20T07:52:38.087 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-paste. 2021-02-20T07:52:38.096 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-paste. 2021-02-20T07:52:38.103 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T07:52:38.110 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-02-20T07:52:38.145 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T07:52:38.154 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T07:52:38.448 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T07:52:38.461 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T07:52:38.474 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-02-20T07:52:38.490 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-02-20T07:52:38.498 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T07:52:38.532 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T07:52:38.792 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T07:52:38.807 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T07:52:38.827 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastedeploy. 2021-02-20T07:52:38.841 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-02-20T07:52:38.850 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T07:52:38.902 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-02-20T07:52:39.154 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T07:52:39.167 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T07:52:39.212 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T07:52:39.213 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webtest. 2021-02-20T07:52:39.226 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-02-20T07:52:39.263 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T07:52:39.573 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T07:52:39.587 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T07:52:39.599 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pecan. 2021-02-20T07:52:39.614 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-02-20T07:52:39.631 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T07:52:39.657 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-02-20T07:52:40.003 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T07:52:40.018 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T07:52:40.037 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-werkzeug. 2021-02-20T07:52:40.053 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-02-20T07:52:40.061 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T07:52:40.096 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T07:52:40.406 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T07:52:40.421 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:40.464 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:40.482 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr. 2021-02-20T07:52:40.497 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../046-ceph-mgr_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:40.541 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:40.909 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T07:52:40.925 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:40.967 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:41.027 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mon. 2021-02-20T07:52:41.040 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../047-ceph-mon_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:41.077 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:41.963 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T07:52:41.978 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:42.021 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:42.050 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-osd. 2021-02-20T07:52:42.064 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../048-ceph-osd_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:42.108 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:44.095 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph. 2021-02-20T07:52:44.111 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../049-ceph_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:44.154 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:44.166 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph. 2021-02-20T07:52:44.180 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../049-ceph_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:44.224 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:44.473 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T07:52:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:44.531 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:44.535 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-fuse. 2021-02-20T07:52:44.551 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../050-ceph-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:44.593 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:44.935 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T07:52:44.948 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:44.985 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:45.005 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mds. 2021-02-20T07:52:45.018 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../051-ceph-mds_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:45.064 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package cephadm. 2021-02-20T07:52:45.494 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../052-cephadm_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:45.538 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:45.567 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package cephadm. 2021-02-20T07:52:45.580 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../052-cephadm_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:52:45.650 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:45.883 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T07:52:45.896 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T07:52:45.932 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T07:52:45.969 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jinja2. 2021-02-20T07:52:45.985 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-02-20T07:52:46.027 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T07:52:46.252 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T07:52:46.266 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:46.310 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:46.342 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-02-20T07:52:46.355 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:46.400 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:46.728 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T07:52:46.743 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T07:52:46.786 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T07:52:46.991 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-repoze.lru. 2021-02-20T07:52:47.007 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-02-20T07:52:47.049 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-02-20T07:52:47.099 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-routes. 2021-02-20T07:52:47.115 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T07:52:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T07:52:47.352 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-routes. 2021-02-20T07:52:47.366 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-02-20T07:52:47.413 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-02-20T07:52:47.750 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T07:52:47.765 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:47.794 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-02-20T07:52:47.808 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:47.852 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:48.663 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T07:52:48.679 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T07:52:48.716 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-numpy. 2021-02-20T07:52:48.721 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T07:52:48.732 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-02-20T07:52:48.774 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T07:52:49.327 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T07:52:49.344 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T07:52:49.386 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T07:52:49.397 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-decorator. 2021-02-20T07:52:49.410 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-02-20T07:52:49.455 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-02-20T07:52:49.747 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T07:52:49.760 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T07:52:49.805 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T07:52:49.825 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-scipy. 2021-02-20T07:52:49.838 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-02-20T07:52:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T07:52:51.081 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T07:52:51.096 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T07:52:51.140 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T07:52:51.185 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-02-20T07:52:51.202 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-02-20T07:52:51.244 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-02-20T07:52:51.604 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T07:52:51.618 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T07:52:51.662 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T07:52:51.726 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-joblib. 2021-02-20T07:52:51.740 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-02-20T07:52:51.785 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-joblib (0.11-1) ... 2021-02-20T07:52:52.069 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T07:52:52.083 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T07:52:52.128 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T07:52:52.139 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sklearn. 2021-02-20T07:52:52.153 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-02-20T07:52:52.189 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-02-20T07:52:52.660 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T07:52:52.677 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:52.719 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:52.784 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-02-20T07:52:52.797 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:52.842 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:53.834 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T07:52:53.848 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T07:52:53.894 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T07:52:53.957 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-monotonic. 2021-02-20T07:52:53.971 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-02-20T07:52:54.015 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-02-20T07:52:54.222 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T07:52:54.236 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T07:52:54.280 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T07:52:54.318 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-fasteners. 2021-02-20T07:52:54.332 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-02-20T07:52:54.376 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-02-20T07:52:54.616 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T07:52:54.631 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T07:52:54.683 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T07:52:54.712 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-httplib2. 2021-02-20T07:52:54.727 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2021-02-20T07:52:54.770 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T07:52:55.003 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T07:52:55.016 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T07:52:55.061 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T07:52:55.082 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1. 2021-02-20T07:52:55.095 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-02-20T07:52:55.140 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-02-20T07:52:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T07:52:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T07:52:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T07:52:55.486 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-02-20T07:52:55.500 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-02-20T07:52:55.544 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T07:52:55.784 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T07:52:55.797 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T07:52:55.834 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T07:52:55.855 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-rsa. 2021-02-20T07:52:55.870 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-02-20T07:52:55.914 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-02-20T07:52:56.153 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T07:52:56.167 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T07:52:56.212 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T07:52:56.216 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-02-20T07:52:56.233 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-02-20T07:52:56.275 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T07:52:56.683 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T07:52:56.698 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T07:52:56.729 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-oauth2client. 2021-02-20T07:52:56.741 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T07:52:56.743 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-02-20T07:52:56.787 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-02-20T07:52:57.052 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T07:52:57.066 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T07:52:57.111 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T07:52:57.124 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-setuptools. 2021-02-20T07:52:57.139 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-02-20T07:52:57.199 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-02-20T07:52:57.489 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T07:52:57.505 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T07:52:57.547 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T07:52:57.586 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-websocket. 2021-02-20T07:52:57.600 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-02-20T07:52:57.644 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T07:52:57.859 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T07:52:57.874 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T07:52:57.909 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T07:52:57.964 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-kubernetes. 2021-02-20T07:52:57.980 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-02-20T07:52:58.022 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T07:52:58.321 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T07:52:58.335 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:58.379 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:58.451 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-02-20T07:52:58.467 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:58.509 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:58.715 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T07:52:58.730 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T07:52:58.773 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T07:52:58.854 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-json-pointer. 2021-02-20T07:52:58.868 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-02-20T07:52:58.937 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-02-20T07:52:59.110 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T07:52:59.127 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T07:52:59.180 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T07:52:59.307 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-jsonpatch. 2021-02-20T07:52:59.320 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-02-20T07:52:59.368 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T07:52:59.473 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T07:52:59.489 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:59.532 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:59.704 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-02-20T07:52:59.719 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../079-ceph-mgr-rook_17.0.0-929-gd318d42d-1bionic_all.deb ... 2021-02-20T07:52:59.771 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:52:59.918 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T07:52:59.932 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T07:53:00.062 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T07:53:00.107 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libonig4:amd64. 2021-02-20T07:53:00.122 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-02-20T07:53:00.243 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-02-20T07:53:00.406 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T07:53:00.420 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T07:53:00.464 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T07:53:00.595 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libjq1:amd64. 2021-02-20T07:53:00.611 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-02-20T07:53:00.654 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T07:53:00.726 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package jq. 2021-02-20T07:53:00.742 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T07:53:00.784 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T07:53:00.956 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package jq. 2021-02-20T07:53:00.971 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-02-20T07:53:01.015 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-02-20T07:53:01.062 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package socat. 2021-02-20T07:53:01.075 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T07:53:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T07:53:01.276 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package socat. 2021-02-20T07:53:01.291 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-02-20T07:53:01.334 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-02-20T07:53:01.523 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T07:53:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T07:53:01.582 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T07:53:01.721 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-02-20T07:53:01.735 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-02-20T07:53:01.779 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T07:53:01.901 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T07:53:01.915 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T07:53:01.959 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T07:53:02.074 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package xmlstarlet. 2021-02-20T07:53:02.089 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-02-20T07:53:02.132 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-02-20T07:53:02.347 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-test. 2021-02-20T07:53:02.362 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../086-ceph-test_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:02.405 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:02.495 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package ceph-test. 2021-02-20T07:53:02.514 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../086-ceph-test_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:02.553 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:05.760 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T07:53:05.776 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:05.835 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:05.900 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package libcephfs-dev. 2021-02-20T07:53:05.916 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../087-libcephfs-dev_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:05.958 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:06.146 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T07:53:06.160 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T07:53:06.204 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T07:53:06.252 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package nvme-cli. 2021-02-20T07:53:06.267 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-02-20T07:53:06.310 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T07:53:06.558 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-attr. 2021-02-20T07:53:06.572 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T07:53:06.616 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T07:53:06.640 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-attr. 2021-02-20T07:53:06.653 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-02-20T07:53:06.690 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-attr (17.4.0-2) ... 2021-02-20T07:53:06.919 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T07:53:06.934 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T07:53:06.977 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T07:53:07.018 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-funcsigs. 2021-02-20T07:53:07.032 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-02-20T07:53:07.076 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-02-20T07:53:07.280 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T07:53:07.295 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T07:53:07.338 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T07:53:07.405 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pluggy. 2021-02-20T07:53:07.419 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-02-20T07:53:07.471 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-02-20T07:53:07.650 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-py. 2021-02-20T07:53:07.668 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T07:53:07.709 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T07:53:07.799 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-py. 2021-02-20T07:53:07.815 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-02-20T07:53:07.857 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-py (1.5.2-1) ... 2021-02-20T07:53:08.045 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-pytest. 2021-02-20T07:53:08.059 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T07:53:08.103 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T07:53:08.169 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python-pytest. 2021-02-20T07:53:08.186 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-02-20T07:53:08.227 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-02-20T07:53:08.448 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-attr. 2021-02-20T07:53:08.463 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T07:53:08.507 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T07:53:08.564 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-attr. 2021-02-20T07:53:08.577 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-02-20T07:53:08.630 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-02-20T07:53:08.810 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T07:53:08.825 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T07:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T07:53:08.976 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-cycler. 2021-02-20T07:53:08.991 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-02-20T07:53:09.042 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-02-20T07:53:09.204 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T07:53:09.217 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T07:53:09.254 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T07:53:09.353 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-webencodings. 2021-02-20T07:53:09.368 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-02-20T07:53:09.411 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-02-20T07:53:09.540 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T07:53:09.557 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T07:53:09.607 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T07:53:09.715 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-html5lib. 2021-02-20T07:53:09.732 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-02-20T07:53:09.773 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-02-20T07:53:09.986 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T07:53:10.000 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T07:53:10.044 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T07:53:10.135 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-02-20T07:53:10.151 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-02-20T07:53:10.193 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T07:53:10.481 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T07:53:10.497 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T07:53:10.539 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T07:53:10.664 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyparsing. 2021-02-20T07:53:10.680 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-02-20T07:53:10.721 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T07:53:10.884 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tz. 2021-02-20T07:53:10.898 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T07:53:10.942 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T07:53:11.074 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tz. 2021-02-20T07:53:11.090 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-02-20T07:53:11.133 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tz (2018.3-2) ... 2021-02-20T07:53:11.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T07:53:11.267 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T07:53:11.311 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T07:53:11.419 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-matplotlib. 2021-02-20T07:53:11.428 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-02-20T07:53:11.469 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T07:53:12.025 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T07:53:12.040 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T07:53:12.083 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T07:53:12.091 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-olefile. 2021-02-20T07:53:12.107 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-02-20T07:53:12.149 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-02-20T07:53:12.395 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T07:53:12.408 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T07:53:12.453 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T07:53:12.495 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pastescript. 2021-02-20T07:53:12.509 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-02-20T07:53:12.545 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-02-20T07:53:12.807 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T07:53:12.822 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T07:53:12.865 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T07:53:12.898 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pil:amd64. 2021-02-20T07:53:12.912 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-02-20T07:53:12.956 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T07:53:13.235 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T07:53:13.250 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T07:53:13.293 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T07:53:13.352 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pluggy. 2021-02-20T07:53:13.367 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-02-20T07:53:13.410 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-02-20T07:53:13.580 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-py. 2021-02-20T07:53:13.597 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T07:53:13.638 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T07:53:13.713 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-py. 2021-02-20T07:53:13.727 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-02-20T07:53:13.771 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-py (1.5.2-1) ... 2021-02-20T07:53:13.999 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T07:53:14.014 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T07:53:14.066 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T07:53:14.099 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pyinotify. 2021-02-20T07:53:14.115 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-02-20T07:53:14.157 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-02-20T07:53:14.377 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T07:53:14.391 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T07:53:14.435 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T07:53:14.535 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-pytest. 2021-02-20T07:53:14.551 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-02-20T07:53:14.602 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-02-20T07:53:14.755 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T07:53:14.772 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T07:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T07:53:14.947 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-simplejson. 2021-02-20T07:53:14.963 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-02-20T07:53:15.014 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-02-20T07:53:15.142 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T07:53:15.155 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T07:53:15.200 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T07:53:15.392 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package python3-tk:amd64. 2021-02-20T07:53:15.407 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-02-20T07:53:15.450 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T07:53:15.629 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package radosgw. 2021-02-20T07:53:15.645 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../111-radosgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:15.687 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:15.904 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package radosgw. 2021-02-20T07:53:15.919 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../111-radosgw_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:15.962 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:16.732 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T07:53:16.745 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:16.798 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:16.983 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package rbd-fuse. 2021-02-20T07:53:16.998 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../112-rbd-fuse_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:17.041 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:17.135 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T07:53:17.150 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:17.193 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:17.404 INFO:teuthology.orchestra.run.smithi106.stdout:Selecting previously unselected package rbd-mirror. 2021-02-20T07:53:17.419 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing to unpack .../113-rbd-mirror_17.0.0-929-gd318d42d-1bionic_amd64.deb ... 2021-02-20T07:53:17.462 INFO:teuthology.orchestra.run.smithi106.stdout:Unpacking rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:18.198 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T07:53:18.515 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T07:53:18.605 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-02-20T07:53:18.892 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-02-20T07:53:18.899 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T07:53:19.026 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T07:53:19.243 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-02-20T07:53:19.344 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-02-20T07:53:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T07:53:19.914 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-02-20T07:53:20.013 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T07:53:20.290 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-02-20T07:53:20.297 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T07:53:20.566 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webencodings (0.5-2) ... 2021-02-20T07:53:20.590 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T07:53:20.791 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T07:53:20.850 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-attr (17.4.0-2) ... 2021-02-20T07:53:20.917 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T07:53:21.027 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-02-20T07:53:21.128 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-py (1.5.2-1) ... 2021-02-20T07:53:21.253 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T07:53:21.439 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-02-20T07:53:21.589 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T07:53:21.777 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-02-20T07:53:21.905 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T07:53:22.090 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-02-20T07:53:22.231 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:22.392 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-argparse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:22.549 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T07:53:22.693 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-02-20T07:53:22.852 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T07:53:22.977 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-02-20T07:53:22.978 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T07:53:23.078 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-02-20T07:53:23.914 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T07:53:23.998 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-02-20T07:53:24.299 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T07:53:24.324 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-02-20T07:53:24.583 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T07:53:24.608 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-py (1.5.2-1) ... 2021-02-20T07:53:24.818 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T07:53:24.835 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-02-20T07:53:25.144 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T07:53:25.145 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-02-20T07:53:25.218 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T07:53:25.234 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-02-20T07:53:25.447 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T07:53:25.471 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-02-20T07:53:25.564 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T07:53:25.606 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-02-20T07:53:25.766 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T07:53:25.808 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-02-20T07:53:26.075 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T07:53:26.100 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-02-20T07:53:26.384 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T07:53:26.418 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-02-20T07:53:26.644 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T07:53:26.711 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-02-20T07:53:26.989 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T07:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-02-20T07:53:27.297 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T07:53:27.389 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-02-20T07:53:27.581 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:27.698 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:28.198 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user cephadm....done 2021-02-20T07:53:28.329 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T07:53:28.357 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user cephadm....done 2021-02-20T07:53:28.522 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-02-20T07:53:28.580 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T07:53:28.689 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T07:53:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-02-20T07:53:28.824 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T07:53:28.949 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-02-20T07:53:29.100 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-02-20T07:53:29.158 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T07:53:29.451 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-02-20T07:53:29.460 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T07:53:29.577 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T07:53:29.744 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-02-20T07:53:29.870 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-02-20T07:53:29.921 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:30.197 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T07:53:30.205 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:30.465 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T07:53:30.523 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-monotonic (1.1-2) ... 2021-02-20T07:53:30.574 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T07:53:30.824 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-02-20T07:53:30.908 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T07:53:30.958 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-02-20T07:53:31.260 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T07:53:31.293 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-02-20T07:53:31.349 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T07:53:31.382 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T07:53:31.620 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T07:53:31.639 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-02-20T07:53:31.726 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-02-20T07:53:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-02-20T07:53:31.896 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T07:53:31.997 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-02-20T07:53:32.182 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T07:53:32.323 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2021-02-20T07:53:32.525 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T07:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-02-20T07:53:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T07:53:32.969 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-attr (17.4.0-2) ... 2021-02-20T07:53:33.019 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:33.120 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T07:53:33.270 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-02-20T07:53:33.479 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libjaeger (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:33.513 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T07:53:33.586 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T07:53:33.605 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-02-20T07:53:33.866 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T07:53:34.024 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-02-20T07:53:34.042 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T07:53:34.130 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-02-20T07:53:34.168 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T07:53:34.385 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liboath0 (2.6.1-1) ... 2021-02-20T07:53:34.428 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T07:53:34.529 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T07:53:34.561 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-02-20T07:53:34.687 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-02-20T07:53:34.881 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T07:53:34.946 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-02-20T07:53:34.982 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T07:53:35.074 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-02-20T07:53:35.276 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T07:53:35.458 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-02-20T07:53:35.586 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-02-20T07:53:35.594 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T07:53:35.870 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tz (2018.3-2) ... 2021-02-20T07:53:35.878 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T07:53:36.155 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T07:53:36.171 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-02-20T07:53:36.464 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-02-20T07:53:36.506 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T07:53:36.774 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-joblib (0.11-1) ... 2021-02-20T07:53:36.941 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T07:53:37.142 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-routes (2.4.1-1) ... 2021-02-20T07:53:37.217 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T07:53:37.468 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-02-20T07:53:37.535 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T07:53:37.769 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-02-20T07:53:37.912 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T07:53:38.120 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-02-20T07:53:38.188 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T07:53:38.289 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T07:53:38.514 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-02-20T07:53:38.824 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up jq (1.5+dfsg-2) ... 2021-02-20T07:53:38.950 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-02-20T07:53:39.453 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T07:53:39.747 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T07:53:40.031 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T07:53:40.132 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T07:53:40.156 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-02-20T07:53:40.474 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-02-20T07:53:40.475 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T07:53:40.734 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T07:53:40.792 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-02-20T07:53:40.860 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T07:53:40.918 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-02-20T07:53:40.987 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librados2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:41.104 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T07:53:41.311 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-pytest (3.3.2-2) ... 2021-02-20T07:53:41.389 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:41.489 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:41.571 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-02-20T07:53:41.591 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:41.697 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-02-20T07:53:41.700 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T07:53:41.823 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librados2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:41.949 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-02-20T07:53:42.001 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T07:53:42.250 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:42.302 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T07:53:42.384 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs-dev (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:42.519 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rados (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:42.645 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-02-20T07:53:42.697 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T07:53:42.976 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-02-20T07:53:43.372 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-02-20T07:53:43.516 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T07:53:43.783 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-02-20T07:53:44.330 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T07:53:44.606 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:44.637 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-02-20T07:53:44.715 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:44.835 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:44.986 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:45.177 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T07:53:45.178 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T07:53:45.447 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up librbd1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:45.449 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-02-20T07:53:45.547 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T07:53:45.621 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T07:53:45.653 INFO:teuthology.orchestra.run.smithi106.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T07:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libradosstriper1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:45.872 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cephfs (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:45.968 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T07:53:45.997 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librgw2 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:46.123 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:46.361 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T07:53:46.362 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-02-20T07:53:46.612 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:46.617 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librbd1 (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:46.713 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T07:53:46.751 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-02-20T07:53:46.841 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-02-20T07:53:46.882 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-02-20T07:53:47.031 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:47.132 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:47.188 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-02-20T07:53:47.233 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:47.350 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:Adding group ceph....done 2021-02-20T07:53:47.865 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-modules-core (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:47.991 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-02-20T07:53:48.179 INFO:teuthology.orchestra.run.smithi106.stdout:Adding system user ceph....done 2021-02-20T07:53:48.363 INFO:teuthology.orchestra.run.smithi106.stdout:Setting system user ceph properties....done 2021-02-20T07:53:48.522 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T07:53:48.668 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:48.727 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T07:53:48.802 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up rbd-fuse (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:48.928 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rbd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:49.054 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-common (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:49.377 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout:Adding group ceph....done 2021-02-20T07:53:49.639 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T07:53:49.639 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T07:53:49.891 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user ceph....done 2021-02-20T07:53:50.050 INFO:teuthology.orchestra.run.smithi114.stdout:Setting system user ceph properties....done 2021-02-20T07:53:50.062 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-02-20T07:53:50.262 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T07:53:50.262 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T07:53:50.407 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-02-20T07:53:50.539 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:50.665 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up radosgw (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:50.665 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:50.910 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T07:53:50.910 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-02-20T07:53:50.967 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T07:53:51.250 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:51.283 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up rbd-mirror (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:51.449 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T07:53:51.450 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T07:53:51.476 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T07:53:51.477 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-02-20T07:53:51.718 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-test (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:51.726 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:51.827 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-base (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:51.878 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.129 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-02-20T07:53:52.305 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T07:53:52.305 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T07:53:52.385 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.589 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.598 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T07:53:52.598 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-02-20T07:53:52.706 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.824 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.864 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-diskprediction-local (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:52.991 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-osd (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:53.030 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T07:53:53.031 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T07:53:53.317 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:53.515 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T07:53:53.515 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-02-20T07:53:53.516 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T07:53:53.516 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T07:53:53.785 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-cephadm (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:53.793 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:53.903 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-rook (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:53.919 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:54.021 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mds (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:54.062 INFO:teuthology.orchestra.run.smithi106.stdout:Setting up ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:54.164 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T07:53:54.229 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T07:53:54.230 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-02-20T07:53:54.488 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mon (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:54.524 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T07:53:54.695 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T07:53:54.696 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-02-20T07:53:54.957 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-k8sevents (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:55.074 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-dashboard (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:55.217 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph (17.0.0-929-gd318d42d-1bionic) ... 2021-02-20T07:53:55.343 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-02-20T07:53:55.519 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-02-20T07:53:56.568 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T07:53:56.611 INFO:teuthology.orchestra.run.smithi106.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T07:53:56.669 INFO:teuthology.orchestra.run.smithi106.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T07:53:57.130 INFO:teuthology.orchestra.run.smithi106.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T07:53:57.132 DEBUG:teuthology.parallel:result is None 2021-02-20T07:53:57.572 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-02-20T07:53:57.606 INFO:teuthology.orchestra.run.smithi114.stdout:ureadahead will be reprofiled on next reboot 2021-02-20T07:53:57.647 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-02-20T07:53:58.091 INFO:teuthology.orchestra.run.smithi114.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-02-20T07:53:58.093 DEBUG:teuthology.parallel:result is None 2021-02-20T07:53:58.093 DEBUG:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2021-02-20T07:53:58.150 INFO:teuthology.orchestra.run.smithi106.stdout:Ubuntu 2021-02-20T07:53:58.155 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:53:58.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:53:58.335 DEBUG:teuthology.orchestra.run.smithi106:> dpkg-query -W -f '${Version}' ceph 2021-02-20T07:53:58.359 INFO:teuthology.orchestra.run.smithi106.stdout:17.0.0-929-gd318d42d-1bionic 2021-02-20T07:53:58.360 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:53:58.360 INFO:teuthology.task.install:The correct ceph version 17.0.0-929-gd318d42d-1bionic is installed. 2021-02-20T07:53:58.361 DEBUG:teuthology.orchestra.run.smithi114:> sudo lsb_release -is 2021-02-20T07:53:58.417 INFO:teuthology.orchestra.run.smithi114.stdout:Ubuntu 2021-02-20T07:53:58.423 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-02-20T07:53:58.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:53:58.588 DEBUG:teuthology.orchestra.run.smithi114:> dpkg-query -W -f '${Version}' ceph 2021-02-20T07:53:58.614 INFO:teuthology.orchestra.run.smithi114.stdout:17.0.0-929-gd318d42d-1bionic 2021-02-20T07:53:58.614 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-929-gd318d42d-1bionic 2021-02-20T07:53:58.615 INFO:teuthology.task.install:The correct ceph version 17.0.0-929-gd318d42d-1bionic is installed. 2021-02-20T07:53:58.615 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-02-20T07:53:58.616 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:53:58.616 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T07:53:58.632 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:53:58.633 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-02-20T07:53:58.671 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-02-20T07:53:58.672 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:53:58.672 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T07:53:58.686 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T07:53:58.742 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:53:58.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2021-02-20T07:53:58.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-02-20T07:53:58.815 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-02-20T07:53:58.815 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:53:58.816 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T07:53:58.830 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T07:53:58.887 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:53:58.887 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2021-02-20T07:53:58.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-02-20T07:53:58.959 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T07:53:59.016 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-02-20T07:53:59.017 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2021-02-20T07:53:59.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2021-02-20T07:53:59.034 INFO:tasks.ceph:Disabling ceph logrotate... 2021-02-20T07:53:59.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T07:53:59.078 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2021-02-20T07:53:59.093 INFO:tasks.ceph:Creating extra log directories... 2021-02-20T07:53:59.093 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T07:53:59.134 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-02-20T07:53:59.191 INFO:tasks.ceph:Creating ceph cluster cluster1... 2021-02-20T07:53:59.192 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T07:53:59.193 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5897891', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'targets': {'smithi106.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1mnk0O6+NreOssSFrZ95wpiF8tx9IRDQM3nDni9DDXQbHBttgEFNAoVR5+7i1QGU1IjrmGogPN1ot03XpA9AbaT/KvymYBq2iL5O5eTwSJ6OAPai1vNH2KD91ISBja7+c9LmcoTu69zflvewYGWRD2Av8vnC3JEGqC/c1ROTnFaxOAmIz8OTxmN7XxxRmq/D2QOBvFFdV1MsshGIPmFGWj7MgYSDZoe7TWpjXJo3L1HG9fa79HoOFSrsn/FvBTQ772Y0YluDkmbNsGXUuHB09bchr1XBIQNqbnlZv/Mx2PschkZIBUc2Ob5d3jpWNjvjJVt7pzVutl4PGvip1eWlL', 'smithi114.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDstIYH2/PVs6ZJhqo/CwHh6FWOoxqMGz3sP3ZIantQz95ZbsZA0mJ/1DV7UlYIghJhw3XsASkRnCEPgYH0yPIU7PeG4IyWjvpEbKNNUXjUmPsEENYZL4IIIY6xFkuKZZpWRvS1WKebQmAPuGWyQAXS2E+Xh0WGQIcy2kC2pPEICjmpjjH8dvtN9+i3xDcbcUZ7T/EnJh+KeCtTPQEFs2kdZCP4m4DMG0hiwqlwPWWgnA2QXMLguc4v19RWDzot2lhDXNyRcq9PGghJifN+R5UbgKRssiEd4UxM44CL1MSdo89fSy04L0VpoHgnzeUn9xhYFodHp1p0EPC9laBn5rv/'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,object-map,fast-diff', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T07:53:59.194 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T07:53:59.197 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-02-20T07:53:59.205 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T07:53:59.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T07:53:59.267 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:53:59.268 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T07:53:59.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T07:53:59.311 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T07:53:59.363 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T07:53:59.363 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:53:59.363 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 341 Links: 1 2021-02-20T07:53:59.364 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:53:59.364 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2021-02-20 07:51:19.680520906 +0000 2021-02-20T07:53:59.364 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2021-02-20 07:51:19.588566908 +0000 2021-02-20T07:53:59.364 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2021-02-20 07:51:19.588566908 +0000 2021-02-20T07:53:59.365 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2021-02-20T07:53:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2021-02-20T07:53:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2021-02-20T07:53:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000300947 s, 1.7 MB/s 2021-02-20T07:53:59.377 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T07:53:59.427 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T07:53:59.428 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:53:59.428 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 347 Links: 1 2021-02-20T07:53:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:53:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2021-02-20 07:51:19.912404900 +0000 2021-02-20T07:53:59.429 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2021-02-20 07:51:19.800460902 +0000 2021-02-20T07:53:59.430 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2021-02-20 07:51:19.800460902 +0000 2021-02-20T07:53:59.430 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2021-02-20T07:53:59.437 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2021-02-20T07:53:59.437 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2021-02-20T07:53:59.437 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000373457 s, 1.4 MB/s 2021-02-20T07:53:59.441 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T07:53:59.491 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T07:53:59.491 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:53:59.492 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 353 Links: 1 2021-02-20T07:53:59.492 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:53:59.493 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2021-02-20 07:51:20.028346898 +0000 2021-02-20T07:53:59.493 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2021-02-20 07:51:20.024348897 +0000 2021-02-20T07:53:59.493 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2021-02-20 07:51:20.024348897 +0000 2021-02-20T07:53:59.494 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2021-02-20T07:53:59.499 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2021-02-20T07:53:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2021-02-20T07:53:59.500 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000327068 s, 1.6 MB/s 2021-02-20T07:53:59.504 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T07:53:59.551 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T07:53:59.551 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:53:59.552 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-02-20T07:53:59.552 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:53:59.552 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2021-02-20 07:51:20.280220891 +0000 2021-02-20T07:53:59.553 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2021-02-20 07:51:20.276222892 +0000 2021-02-20T07:53:59.553 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2021-02-20 07:51:20.276222892 +0000 2021-02-20T07:53:59.553 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2021-02-20T07:53:59.561 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2021-02-20T07:53:59.562 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2021-02-20T07:53:59.562 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000381053 s, 1.3 MB/s 2021-02-20T07:53:59.565 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:53:59.566 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T07:53:59.566 INFO:tasks.ceph:Generating config... 2021-02-20T07:53:59.568 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T07:53:59.568 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T07:53:59.568 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T07:53:59.569 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T07:53:59.569 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2021-02-20T07:53:59.570 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:53:59.571 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:53:59.571 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:53:59.572 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:53:59.572 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:53:59.573 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:53:59.573 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:53:59.574 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:53:59.575 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T07:53:59.575 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T07:53:59.576 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T07:53:59.576 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T07:53:59.577 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T07:53:59.577 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T07:53:59.578 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T07:53:59.579 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T07:53:59.579 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2021-02-20T07:53:59.580 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-02-20T07:53:59.580 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T07:53:59.581 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T07:53:59.581 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T07:53:59.582 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T07:53:59.582 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T07:53:59.583 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T07:53:59.584 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T07:53:59.584 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T07:53:59.585 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T07:53:59.585 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T07:53:59.586 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T07:53:59.586 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T07:53:59.587 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T07:53:59.588 INFO:tasks.ceph:Setting up cluster1.mon.a... 2021-02-20T07:53:59.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2021-02-20T07:53:59.642 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/cluster1.keyring 2021-02-20T07:53:59.644 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2021-02-20T07:53:59.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T07:53:59.773 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.106')] 2021-02-20T07:53:59.774 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T07:53:59.775 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:53:59.775 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T07:53:59.826 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --print /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:53:59.902 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:53:59.903 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid ddbb55ee-78b2-4437-96f9-c64c271c28fe 2021-02-20T07:53:59.903 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2021-02-20T07:53:59.903 INFO:teuthology.orchestra.run.smithi106.stdout:fsid ddbb55ee-78b2-4437-96f9-c64c271c28fe 2021-02-20T07:53:59.904 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2021-02-20T07:53:59.903760+0000 2021-02-20T07:53:59.904 INFO:teuthology.orchestra.run.smithi106.stdout:created 2021-02-20T07:53:59.903760+0000 2021-02-20T07:53:59.904 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 0 (unknown) 2021-02-20T07:53:59.904 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2021-02-20T07:53:59.905 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2021-02-20T07:53:59.905 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2021-02-20T07:53:59.906 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T07:53:59.950 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID ddbb55ee-78b2-4437-96f9-c64c271c28fe... 2021-02-20T07:53:59.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T07:53:59.994 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-02-20T07:54:00.017 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2021-02-20T07:54:00.017 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2021-02-20T07:54:00.017 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T07:54:00.017 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2021-02-20T07:54:00.018 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.018 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2021-02-20T07:54:00.018 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.018 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2021-02-20T07:54:00.019 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.019 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2021-02-20T07:54:00.019 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2021-02-20T07:54:00.019 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.019 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2021-02-20T07:54:00.020 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2021-02-20T07:54:00.020 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.020 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T07:54:00.020 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2021-02-20T07:54:00.020 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.021 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T07:54:00.021 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T07:54:00.021 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.021 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2021-02-20T07:54:00.022 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.022 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2021-02-20T07:54:00.023 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2021-02-20T07:54:00.023 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2021-02-20T07:54:00.023 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2021-02-20T07:54:00.023 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2021-02-20T07:54:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2021-02-20T07:54:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2021-02-20T07:54:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T07:54:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T07:54:00.024 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2021-02-20T07:54:00.025 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.025 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T07:54:00.025 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.026 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2021-02-20T07:54:00.026 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.026 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2021-02-20T07:54:00.026 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.026 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2021-02-20T07:54:00.027 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2021-02-20T07:54:00.027 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2021-02-20T07:54:00.027 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = 172.21.15.106 2021-02-20T07:54:00.027 INFO:teuthology.orchestra.run.smithi106.stdout: mon client directed command retry = 5 2021-02-20T07:54:00.028 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject socket failures = 5000 2021-02-20T07:54:00.028 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = ddbb55ee-78b2-4437-96f9-c64c271c28fe 2021-02-20T07:54:00.028 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.028 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2021-02-20T07:54:00.029 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2021-02-20T07:54:00.029 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.029 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2021-02-20T07:54:00.029 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2021-02-20T07:54:00.030 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.030 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2021-02-20T07:54:00.030 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2021-02-20T07:54:00.030 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.031 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2021-02-20T07:54:00.031 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2021-02-20T07:54:00.031 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2021-02-20T07:54:00.031 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.031 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2021-02-20T07:54:00.032 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2021-02-20T07:54:00.032 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.033 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2021-02-20T07:54:00.033 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.033 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2021-02-20T07:54:00.033 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.033 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2021-02-20T07:54:00.034 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.034 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2021-02-20T07:54:00.034 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2021-02-20T07:54:00.034 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.035 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2021-02-20T07:54:00.035 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2021-02-20T07:54:00.035 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2021-02-20T07:54:00.035 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore compression algorithm = snappy 2021-02-20T07:54:00.035 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore compression mode = aggressive 2021-02-20T07:54:00.036 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2021-02-20T07:54:00.036 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2021-02-20T07:54:00.036 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2021-02-20T07:54:00.036 INFO:teuthology.orchestra.run.smithi106.stdout: debug filestore = 20 2021-02-20T07:54:00.036 INFO:teuthology.orchestra.run.smithi106.stdout: debug journal = 20 2021-02-20T07:54:00.037 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 20 2021-02-20T07:54:00.037 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 25 2021-02-20T07:54:00.037 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2021-02-20T07:54:00.037 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T07:54:00.037 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2021-02-20T07:54:00.038 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2021-02-20T07:54:00.038 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2021-02-20T07:54:00.038 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2021-02-20T07:54:00.038 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.039 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2021-02-20T07:54:00.039 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2021-02-20T07:54:00.039 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2021-02-20T07:54:00.039 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2021-02-20T07:54:00.039 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2021-02-20T07:54:00.040 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:00.040 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:00.040 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2021-02-20T07:54:00.040 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.040 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2021-02-20T07:54:00.041 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2021-02-20T07:54:00.041 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2021-02-20T07:54:00.041 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2021-02-20T07:54:00.041 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2021-02-20T07:54:00.042 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2021-02-20T07:54:00.042 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2021-02-20T07:54:00.042 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:00.042 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2021-02-20T07:54:00.042 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2021-02-20T07:54:00.043 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:00.043 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:00.043 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2021-02-20T07:54:00.043 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2021-02-20T07:54:00.044 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2021-02-20T07:54:00.044 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2021-02-20T07:54:00.044 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T07:54:00.044 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T07:54:00.044 INFO:teuthology.orchestra.run.smithi106.stdout: debug rbd = 20 2021-02-20T07:54:00.045 INFO:teuthology.orchestra.run.smithi106.stdout: debug rbd_mirror = 15 2021-02-20T07:54:00.045 INFO:teuthology.orchestra.run.smithi106.stdout: log to stderr = False 2021-02-20T07:54:00.045 INFO:teuthology.orchestra.run.smithi106.stdout: rbd default features = 125 2021-02-20T07:54:00.045 INFO:teuthology.orchestra.run.smithi106.stdout: rbd mirror image policy type = simple 2021-02-20T07:54:00.046 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2021-02-20T07:54:00.046 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.0] 2021-02-20T07:54:00.046 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.046 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.046 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.1] 2021-02-20T07:54:00.047 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.047 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.047 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.2] 2021-02-20T07:54:00.047 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.048 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.048 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.3] 2021-02-20T07:54:00.048 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.049 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.050 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2021-02-20T07:54:00.050 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2021-02-20T07:54:00.051 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T07:54:00.051 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2021-02-20T07:54:00.051 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.051 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2021-02-20T07:54:00.051 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.052 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2021-02-20T07:54:00.052 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.052 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2021-02-20T07:54:00.052 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2021-02-20T07:54:00.053 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.053 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2021-02-20T07:54:00.053 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2021-02-20T07:54:00.053 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.053 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T07:54:00.054 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2021-02-20T07:54:00.054 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.054 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T07:54:00.054 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T07:54:00.054 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.055 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2021-02-20T07:54:00.055 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.055 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2021-02-20T07:54:00.055 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2021-02-20T07:54:00.056 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2021-02-20T07:54:00.056 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2021-02-20T07:54:00.056 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2021-02-20T07:54:00.056 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2021-02-20T07:54:00.056 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2021-02-20T07:54:00.057 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T07:54:00.057 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T07:54:00.057 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2021-02-20T07:54:00.057 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.058 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T07:54:00.058 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.058 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2021-02-20T07:54:00.058 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.058 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2021-02-20T07:54:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2021-02-20T07:54:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2021-02-20T07:54:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2021-02-20T07:54:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = 172.21.15.106 2021-02-20T07:54:00.060 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2021-02-20T07:54:00.060 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 5000 2021-02-20T07:54:00.060 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = ddbb55ee-78b2-4437-96f9-c64c271c28fe 2021-02-20T07:54:00.060 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.061 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2021-02-20T07:54:00.061 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2021-02-20T07:54:00.061 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.061 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2021-02-20T07:54:00.061 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2021-02-20T07:54:00.062 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.062 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2021-02-20T07:54:00.062 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2021-02-20T07:54:00.062 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.063 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2021-02-20T07:54:00.063 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2021-02-20T07:54:00.063 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2021-02-20T07:54:00.063 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.064 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2021-02-20T07:54:00.064 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2021-02-20T07:54:00.064 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.065 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2021-02-20T07:54:00.065 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.065 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2021-02-20T07:54:00.065 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.066 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2021-02-20T07:54:00.066 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.066 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2021-02-20T07:54:00.067 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2021-02-20T07:54:00.067 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.067 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2021-02-20T07:54:00.067 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2021-02-20T07:54:00.067 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2021-02-20T07:54:00.068 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression algorithm = snappy 2021-02-20T07:54:00.068 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression mode = aggressive 2021-02-20T07:54:00.068 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2021-02-20T07:54:00.068 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2021-02-20T07:54:00.068 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2021-02-20T07:54:00.069 INFO:teuthology.orchestra.run.smithi114.stdout: debug filestore = 20 2021-02-20T07:54:00.069 INFO:teuthology.orchestra.run.smithi114.stdout: debug journal = 20 2021-02-20T07:54:00.069 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 20 2021-02-20T07:54:00.069 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 25 2021-02-20T07:54:00.070 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2021-02-20T07:54:00.070 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T07:54:00.070 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2021-02-20T07:54:00.070 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2021-02-20T07:54:00.070 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2021-02-20T07:54:00.071 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2021-02-20T07:54:00.071 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.071 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2021-02-20T07:54:00.071 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2021-02-20T07:54:00.072 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2021-02-20T07:54:00.072 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2021-02-20T07:54:00.072 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2021-02-20T07:54:00.072 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:00.072 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:00.073 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2021-02-20T07:54:00.073 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.073 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2021-02-20T07:54:00.073 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2021-02-20T07:54:00.074 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2021-02-20T07:54:00.074 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2021-02-20T07:54:00.074 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2021-02-20T07:54:00.074 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2021-02-20T07:54:00.074 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2021-02-20T07:54:00.075 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:00.075 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2021-02-20T07:54:00.075 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2021-02-20T07:54:00.075 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:00.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:00.076 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2021-02-20T07:54:00.076 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2021-02-20T07:54:00.076 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2021-02-20T07:54:00.076 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2021-02-20T07:54:00.077 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T07:54:00.077 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T07:54:00.077 INFO:teuthology.orchestra.run.smithi114.stdout: debug rbd = 20 2021-02-20T07:54:00.077 INFO:teuthology.orchestra.run.smithi114.stdout: debug rbd_mirror = 15 2021-02-20T07:54:00.077 INFO:teuthology.orchestra.run.smithi114.stdout: log to stderr = False 2021-02-20T07:54:00.078 INFO:teuthology.orchestra.run.smithi114.stdout: rbd default features = 125 2021-02-20T07:54:00.078 INFO:teuthology.orchestra.run.smithi114.stdout: rbd mirror image policy type = simple 2021-02-20T07:54:00.078 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2021-02-20T07:54:00.078 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.0] 2021-02-20T07:54:00.078 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.079 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.079 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.1] 2021-02-20T07:54:00.079 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.079 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.080 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.2] 2021-02-20T07:54:00.080 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.080 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.080 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.3] 2021-02-20T07:54:00.080 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:00.081 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:00.081 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2021-02-20T07:54:00.081 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2021-02-20T07:54:00.136 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T07:54:00.136 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:00.137 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2021-02-20T07:54:00.182 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:00.183 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2021-02-20T07:54:00.230 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:54:00.231 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:00.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T07:54:00.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T07:54:00.297 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:00.297 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:00.346 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:54:00.347 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:00.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/cluster1.keyring 2021-02-20T07:54:00.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-02-20T07:54:00.373 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:00.373 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:00.427 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T07:54:00.428 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T07:54:00.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T07:54:00.468 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2021-02-20T07:54:00.471 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T07:54:00.471 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T07:54:00.472 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2021-02-20T07:54:00.545 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/cluster1.client.0.keyring 2021-02-20T07:54:00.557 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T07:54:00.592 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2021-02-20T07:54:00.604 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T07:54:00.681 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2021-02-20T07:54:00.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T07:54:00.769 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2021-02-20T07:54:00.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T07:54:00.857 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2021-02-20T07:54:00.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T07:54:00.945 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2021-02-20T07:54:00.957 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T07:54:01.033 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2021-02-20T07:54:01.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T07:54:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2021-02-20T07:54:01.133 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T07:54:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2021-02-20T07:54:01.221 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T07:54:01.222 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T07:54:01.222 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2021-02-20T07:54:01.237 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:01.238 INFO:tasks.ceph:role: cluster1.osd.0 2021-02-20T07:54:01.238 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:54:01.238 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T07:54:01.767 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:01.767 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:01.768 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:01.768 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:01.768 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:01.768 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:01.768 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:01.769 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:01.769 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:01.770 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2021-02-20T07:54:01.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2021-02-20T07:54:01.839 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2021-02-20T07:54:01.852 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:01.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:01.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2021-02-20T07:54:01.914 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:01.914 INFO:tasks.ceph:role: cluster1.osd.1 2021-02-20T07:54:01.915 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:54:01.915 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T07:54:02.448 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:02.448 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:02.449 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:02.449 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:02.449 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:02.449 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:02.450 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:02.450 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:02.450 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:02.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2021-02-20T07:54:02.452 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2021-02-20T07:54:02.501 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2021-02-20T07:54:02.511 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:02.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:02.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2021-02-20T07:54:02.567 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:02.567 INFO:tasks.ceph:role: cluster1.osd.2 2021-02-20T07:54:02.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:54:02.568 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T07:54:03.076 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:03.076 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:03.077 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:03.077 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:03.077 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:03.077 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:03.078 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:03.078 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:03.078 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:03.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2021-02-20T07:54:03.080 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2021-02-20T07:54:03.136 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2021-02-20T07:54:03.148 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:03.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:03.150 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:03.235 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.235+0000 7f04db149080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2021-02-20T07:54:03.236 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.235+0000 7f04db149080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2021-02-20T07:54:03.236 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.235+0000 7f04db149080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T07:54:03.237 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.235+0000 7f04db149080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T07:54:03.237 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.235+0000 7f04db149080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-02-20T07:54:03.239 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:03.239+0000 7f04db149080 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2021-02-20T07:54:06.189 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2021-02-20T07:54:06.248 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.327+0000 7fe2796b1080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2021-02-20T07:54:06.329 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.331+0000 7fe2796b1080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2021-02-20T07:54:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.331+0000 7fe2796b1080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T07:54:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.331+0000 7fe2796b1080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T07:54:06.330 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.331+0000 7fe2796b1080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-02-20T07:54:06.331 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:06.331+0000 7fe2796b1080 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2021-02-20T07:54:09.308 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2021-02-20T07:54:09.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:09.447 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2021-02-20T07:54:09.447 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2021-02-20T07:54:09.447 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T07:54:09.448 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T07:54:09.448 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-02-20T07:54:09.448 INFO:teuthology.orchestra.run.smithi106.stderr:2021-02-20T07:54:09.447+0000 7f7ff2dd9080 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2021-02-20T07:54:12.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2021-02-20T07:54:12.482 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T07:54:12.483 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:12.483 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2021-02-20T07:54:12.539 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:12.539 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2021-02-20T07:54:12.595 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:12.596 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2021-02-20T07:54:12.650 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:12.650 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2021-02-20T07:54:12.706 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:12.707 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2021-02-20T07:54:12.754 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.755 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2021-02-20T07:54:12.761 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.762 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2021-02-20T07:54:12.810 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.810 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2021-02-20T07:54:12.858 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.859 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2021-02-20T07:54:12.906 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.906 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2021-02-20T07:54:12.954 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:12.955 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2021-02-20T07:54:13.005 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:13.006 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2021-02-20T07:54:13.054 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:13.054 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2021-02-20T07:54:13.102 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T07:54:13.102 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/cluster1.keyring 2021-02-20T07:54:13.116 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2021-02-20T07:54:13.116 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBgpo4cHBAACoVS4NUicFncN/tOWoYl9Q== 2021-02-20T07:54:13.116 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2021-02-20T07:54:13.116 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAbwDBgKfE4DhAA3TV64b+fNuBynj0R2yy20A== 2021-02-20T07:54:13.117 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2021-02-20T07:54:13.117 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAewDBg/XPGExAAffBLAOylu9hsOJOSeQjrQQ== 2021-02-20T07:54:13.117 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2021-02-20T07:54:13.117 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAhwDBgZ2PLGhAAzi/tS+aWVWYPDDg12DJCgg== 2021-02-20T07:54:13.118 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2021-02-20T07:54:13.118 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBgu6isIBAAUFCAVEU4/2eSy3p9USy7YQ== 2021-02-20T07:54:13.118 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror] 2021-02-20T07:54:13.118 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBg2pZ3IxAA7RbLrntGiT1cRAEcRNvDGQ== 2021-02-20T07:54:13.118 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.0] 2021-02-20T07:54:13.119 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBg20m/KBAAJJwZYR7V8nZ56eKgXVYwzA== 2021-02-20T07:54:13.119 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.1] 2021-02-20T07:54:13.119 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBgrdMDLhAAm4/7AjKVEuRgHwJXqrQ2pw== 2021-02-20T07:54:13.119 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.2] 2021-02-20T07:54:13.120 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBg6cY+MxAA27IM0wRaUAaB+VLa285cIw== 2021-02-20T07:54:13.120 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.3] 2021-02-20T07:54:13.120 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAYwDBggsB/OBAAmbrRs1jGeiHZG69tZMF63Q== 2021-02-20T07:54:13.120 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.4] 2021-02-20T07:54:13.120 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAZwDBggVUgAhAAWNMgkKd/o/Wt4Yg4bsW7IQ== 2021-02-20T07:54:13.121 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.5] 2021-02-20T07:54:13.121 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAZwDBg+P9aBxAA3OdmXH8Ifrg5QcsxZZYRtQ== 2021-02-20T07:54:13.121 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.6] 2021-02-20T07:54:13.121 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAZwDBg5umdDBAAp/q5BRkTqUVaodXsok9Klg== 2021-02-20T07:54:13.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T07:54:13.206 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:13.289 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:13.375 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:13.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.544 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.677 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.764 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.848 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:13.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:14.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:14.105 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T07:54:14.105 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2021-02-20T07:54:14.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2021-02-20T07:54:14.274 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2021-02-20T07:54:14.291 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2021-02-20T07:54:14.338 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2021-02-20T07:54:14.338 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2021-02-20T07:54:14.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2021-02-20T07:54:14.382 INFO:tasks.ceph.cluster1.mon.a:Started 2021-02-20T07:54:14.382 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2021-02-20T07:54:14.383 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2021-02-20T07:54:14.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2021-02-20T07:54:14.386 INFO:tasks.ceph.cluster1.mgr.x:Started 2021-02-20T07:54:14.386 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T07:54:14.387 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd crush tunables default 2021-02-20T07:54:14.791 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2021-02-20T07:54:14.803 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2021-02-20T07:54:14.804 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:14.804 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2021-02-20T07:54:14.816 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:14.816 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2021-02-20T07:54:14.874 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:14.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2021-02-20T07:54:14.930 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd new 38e73e01-daf8-4b2f-b901-30edde61b2b9 0 2021-02-20T07:54:15.184 INFO:teuthology.orchestra.run.smithi106.stdout:0 2021-02-20T07:54:15.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd new 5a1ee0d0-d2d9-4a8e-9265-14852e278222 1 2021-02-20T07:54:15.413 INFO:teuthology.orchestra.run.smithi106.stdout:1 2021-02-20T07:54:15.426 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd new 79135c80-e640-4a75-92c2-986a371ea699 2 2021-02-20T07:54:15.633 INFO:teuthology.orchestra.run.smithi106.stdout:2 2021-02-20T07:54:15.646 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2021-02-20T07:54:15.647 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2021-02-20T07:54:15.649 INFO:tasks.ceph.cluster1.osd.0:Started 2021-02-20T07:54:15.650 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2021-02-20T07:54:15.650 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2021-02-20T07:54:15.652 INFO:tasks.ceph.cluster1.osd.1:Started 2021-02-20T07:54:15.653 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2021-02-20T07:54:15.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2021-02-20T07:54:15.655 INFO:tasks.ceph.cluster1.osd.2:Started 2021-02-20T07:54:15.656 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T07:54:15.959 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T07:54:16.153 INFO:teuthology.misc.health.smithi106.stdout: 2021-02-20T07:54:16.154 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":5,"fsid":"ddbb55ee-78b2-4437-96f9-c64c271c28fe","created":"2021-02-20T07:54:14.499602+0000","modified":"2021-02-20T07:54:15.633679+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"38e73e01-daf8-4b2f-b901-30edde61b2b9","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a1ee0d0-d2d9-4a8e-9265-14852e278222","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"79135c80-e640-4a75-92c2-986a371ea699","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:16.164 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T07:54:16.269 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T07:54:16.271+0000 7f62dec56080 -1 Falling back to public interface 2021-02-20T07:54:16.270 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:54:16.271+0000 7f50a4db8080 -1 Falling back to public interface 2021-02-20T07:54:16.282 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T07:54:16.283+0000 7f1c8f242080 -1 Falling back to public interface 2021-02-20T07:54:20.006 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:54:20.007+0000 7f50a4db8080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T07:54:20.027 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T07:54:20.027+0000 7f62dec56080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T07:54:20.071 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T07:54:20.071+0000 7f1c8f242080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T07:54:21.492 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:54:21.495+0000 7f509c281700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T07:54:21.493 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T07:54:21.495+0000 7f62d611f700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T07:54:21.493 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T07:54:21.495+0000 7f1c8670b700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T07:54:22.468 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-02-20T07:54:22.763 INFO:teuthology.misc.health.smithi106.stdout: 2021-02-20T07:54:22.764 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":8,"fsid":"ddbb55ee-78b2-4437-96f9-c64c271c28fe","created":"2021-02-20T07:54:14.499602+0000","modified":"2021-02-20T07:54:22.488098+0000","last_up_change":"2021-02-20T07:54:22.488098+0000","last_in_change":"2021-02-20T07:54:22.488098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"38e73e01-daf8-4b2f-b901-30edde61b2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":15600},{"type":"v1","addr":"172.21.15.106:6817","nonce":15600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":15600},{"type":"v1","addr":"172.21.15.106:6819","nonce":15600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":15600},{"type":"v1","addr":"172.21.15.106:6823","nonce":15600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":15600},{"type":"v1","addr":"172.21.15.106:6821","nonce":15600}]},"public_addr":"172.21.15.106:6817/15600","cluster_addr":"172.21.15.106:6819/15600","heartbeat_back_addr":"172.21.15.106:6823/15600","heartbeat_front_addr":"172.21.15.106:6821/15600","state":["exists","up"]},{"osd":1,"uuid":"5a1ee0d0-d2d9-4a8e-9265-14852e278222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":15601},{"type":"v1","addr":"172.21.15.106:6801","nonce":15601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":15601},{"type":"v1","addr":"172.21.15.106:6803","nonce":15601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":15601},{"type":"v1","addr":"172.21.15.106:6810","nonce":15601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":15601},{"type":"v1","addr":"172.21.15.106:6806","nonce":15601}]},"public_addr":"172.21.15.106:6801/15601","cluster_addr":"172.21.15.106:6803/15601","heartbeat_back_addr":"172.21.15.106:6810/15601","heartbeat_front_addr":"172.21.15.106:6806/15601","state":["exists","up"]},{"osd":2,"uuid":"79135c80-e640-4a75-92c2-986a371ea699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6805","nonce":15602},{"type":"v1","addr":"172.21.15.106:6807","nonce":15602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":15602},{"type":"v1","addr":"172.21.15.106:6811","nonce":15602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":15602},{"type":"v1","addr":"172.21.15.106:6815","nonce":15602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":15602},{"type":"v1","addr":"172.21.15.106:6813","nonce":15602}]},"public_addr":"172.21.15.106:6807/15602","cluster_addr":"172.21.15.106:6811/15602","heartbeat_back_addr":"172.21.15.106:6815/15602","heartbeat_front_addr":"172.21.15.106:6813/15602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:22.777 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T07:54:22.777 INFO:tasks.ceph:Creating RBD pool 2021-02-20T07:54:22.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd pool create rbd 8 2021-02-20T07:54:23.506 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'rbd' created 2021-02-20T07:54:23.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T07:54:24.525 INFO:teuthology.orchestra.run.smithi106.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T07:54:24.538 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2021-02-20T07:54:24.539 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T07:54:24.540 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T07:54:24.832 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:24.832 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"ddbb55ee-78b2-4437-96f9-c64c271c28fe","created":"2021-02-20T07:54:14.499602+0000","modified":"2021-02-20T07:54:24.500825+0000","last_up_change":"2021-02-20T07:54:22.488098+0000","last_in_change":"2021-02-20T07:54:22.488098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:22.554396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:23.144894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"38e73e01-daf8-4b2f-b901-30edde61b2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":15600},{"type":"v1","addr":"172.21.15.106:6817","nonce":15600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":15600},{"type":"v1","addr":"172.21.15.106:6819","nonce":15600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":15600},{"type":"v1","addr":"172.21.15.106:6823","nonce":15600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":15600},{"type":"v1","addr":"172.21.15.106:6821","nonce":15600}]},"public_addr":"172.21.15.106:6817/15600","cluster_addr":"172.21.15.106:6819/15600","heartbeat_back_addr":"172.21.15.106:6823/15600","heartbeat_front_addr":"172.21.15.106:6821/15600","state":["exists","up"]},{"osd":1,"uuid":"5a1ee0d0-d2d9-4a8e-9265-14852e278222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":15601},{"type":"v1","addr":"172.21.15.106:6801","nonce":15601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":15601},{"type":"v1","addr":"172.21.15.106:6803","nonce":15601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":15601},{"type":"v1","addr":"172.21.15.106:6810","nonce":15601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":15601},{"type":"v1","addr":"172.21.15.106:6806","nonce":15601}]},"public_addr":"172.21.15.106:6801/15601","cluster_addr":"172.21.15.106:6803/15601","heartbeat_back_addr":"172.21.15.106:6810/15601","heartbeat_front_addr":"172.21.15.106:6806/15601","state":["exists","up"]},{"osd":2,"uuid":"79135c80-e640-4a75-92c2-986a371ea699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6805","nonce":15602},{"type":"v1","addr":"172.21.15.106:6807","nonce":15602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":15602},{"type":"v1","addr":"172.21.15.106:6811","nonce":15602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":15602},{"type":"v1","addr":"172.21.15.106:6815","nonce":15602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":15602},{"type":"v1","addr":"172.21.15.106:6813","nonce":15602}]},"public_addr":"172.21.15.106:6807/15602","cluster_addr":"172.21.15.106:6811/15602","heartbeat_back_addr":"172.21.15.106:6815/15602","heartbeat_front_addr":"172.21.15.106:6813/15602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.038609+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.011228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.058108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:24.849 INFO:tasks.ceph.ceph_manager.cluster1:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T07:54:22.554396+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T07:54:23.144894+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T07:54:24.850 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get device_health_metrics pg_num 2021-02-20T07:54:25.152 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2021-02-20T07:54:25.167 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get rbd pg_num 2021-02-20T07:54:25.470 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 8 2021-02-20T07:54:25.484 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2021-02-20T07:54:25.484 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2021-02-20T07:54:25.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2021-02-20T07:54:25.818 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":15416},{"type":"v1","addr":"172.21.15.106:6825","nonce":15416}]},"active_addr":"172.21.15.106:6825/15416","active_change":"2021-02-20T07:54:19.244798+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2989854603}]},{"addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":83599672}]}]}} 2021-02-20T07:54:25.841 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2021-02-20T07:54:25.842 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2021-02-20T07:54:25.843 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T07:54:26.140 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:26.140 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":11,"fsid":"ddbb55ee-78b2-4437-96f9-c64c271c28fe","created":"2021-02-20T07:54:14.499602+0000","modified":"2021-02-20T07:54:25.516085+0000","last_up_change":"2021-02-20T07:54:22.488098+0000","last_in_change":"2021-02-20T07:54:22.488098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:22.554396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:23.144894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"38e73e01-daf8-4b2f-b901-30edde61b2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":15600},{"type":"v1","addr":"172.21.15.106:6817","nonce":15600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":15600},{"type":"v1","addr":"172.21.15.106:6819","nonce":15600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":15600},{"type":"v1","addr":"172.21.15.106:6823","nonce":15600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":15600},{"type":"v1","addr":"172.21.15.106:6821","nonce":15600}]},"public_addr":"172.21.15.106:6817/15600","cluster_addr":"172.21.15.106:6819/15600","heartbeat_back_addr":"172.21.15.106:6823/15600","heartbeat_front_addr":"172.21.15.106:6821/15600","state":["exists","up"]},{"osd":1,"uuid":"5a1ee0d0-d2d9-4a8e-9265-14852e278222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":15601},{"type":"v1","addr":"172.21.15.106:6801","nonce":15601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":15601},{"type":"v1","addr":"172.21.15.106:6803","nonce":15601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":15601},{"type":"v1","addr":"172.21.15.106:6810","nonce":15601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":15601},{"type":"v1","addr":"172.21.15.106:6806","nonce":15601}]},"public_addr":"172.21.15.106:6801/15601","cluster_addr":"172.21.15.106:6803/15601","heartbeat_back_addr":"172.21.15.106:6810/15601","heartbeat_front_addr":"172.21.15.106:6806/15601","state":["exists","up"]},{"osd":2,"uuid":"79135c80-e640-4a75-92c2-986a371ea699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6805","nonce":15602},{"type":"v1","addr":"172.21.15.106:6807","nonce":15602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":15602},{"type":"v1","addr":"172.21.15.106:6811","nonce":15602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":15602},{"type":"v1","addr":"172.21.15.106:6815","nonce":15602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":15602},{"type":"v1","addr":"172.21.15.106:6813","nonce":15602}]},"public_addr":"172.21.15.106:6807/15602","cluster_addr":"172.21.15.106:6811/15602","heartbeat_back_addr":"172.21.15.106:6815/15602","heartbeat_front_addr":"172.21.15.106:6813/15602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.038609+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.011228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.058108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:26.155 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2021-02-20T07:54:26.156 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-02-20T07:54:26.451 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:26.451 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":11,"fsid":"ddbb55ee-78b2-4437-96f9-c64c271c28fe","created":"2021-02-20T07:54:14.499602+0000","modified":"2021-02-20T07:54:25.516085+0000","last_up_change":"2021-02-20T07:54:22.488098+0000","last_in_change":"2021-02-20T07:54:22.488098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:22.554396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:23.144894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"38e73e01-daf8-4b2f-b901-30edde61b2b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":15600},{"type":"v1","addr":"172.21.15.106:6817","nonce":15600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":15600},{"type":"v1","addr":"172.21.15.106:6819","nonce":15600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":15600},{"type":"v1","addr":"172.21.15.106:6823","nonce":15600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":15600},{"type":"v1","addr":"172.21.15.106:6821","nonce":15600}]},"public_addr":"172.21.15.106:6817/15600","cluster_addr":"172.21.15.106:6819/15600","heartbeat_back_addr":"172.21.15.106:6823/15600","heartbeat_front_addr":"172.21.15.106:6821/15600","state":["exists","up"]},{"osd":1,"uuid":"5a1ee0d0-d2d9-4a8e-9265-14852e278222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":15601},{"type":"v1","addr":"172.21.15.106:6801","nonce":15601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":15601},{"type":"v1","addr":"172.21.15.106:6803","nonce":15601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":15601},{"type":"v1","addr":"172.21.15.106:6810","nonce":15601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":15601},{"type":"v1","addr":"172.21.15.106:6806","nonce":15601}]},"public_addr":"172.21.15.106:6801/15601","cluster_addr":"172.21.15.106:6803/15601","heartbeat_back_addr":"172.21.15.106:6810/15601","heartbeat_front_addr":"172.21.15.106:6806/15601","state":["exists","up"]},{"osd":2,"uuid":"79135c80-e640-4a75-92c2-986a371ea699","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6805","nonce":15602},{"type":"v1","addr":"172.21.15.106:6807","nonce":15602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6809","nonce":15602},{"type":"v1","addr":"172.21.15.106:6811","nonce":15602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":15602},{"type":"v1","addr":"172.21.15.106:6815","nonce":15602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":15602},{"type":"v1","addr":"172.21.15.106:6813","nonce":15602}]},"public_addr":"172.21.15.106:6807/15602","cluster_addr":"172.21.15.106:6811/15602","heartbeat_back_addr":"172.21.15.106:6815/15602","heartbeat_front_addr":"172.21.15.106:6813/15602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.038609+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.011228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:21.058108+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:26.465 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2021-02-20T07:54:26.642 INFO:teuthology.orchestra.run.smithi106.stdout:34359738371 2021-02-20T07:54:26.643 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2021-02-20T07:54:26.816 INFO:teuthology.orchestra.run.smithi106.stdout:34359738370 2021-02-20T07:54:26.817 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2021-02-20T07:54:26.985 INFO:teuthology.orchestra.run.smithi106.stdout:34359738371 2021-02-20T07:54:26.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T07:54:27.285 INFO:teuthology.orchestra.run.smithi106.stdout:34359738370 2021-02-20T07:54:27.299 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738370 for osd.0 2021-02-20T07:54:28.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-02-20T07:54:28.609 INFO:teuthology.orchestra.run.smithi106.stdout:34359738371 2021-02-20T07:54:28.623 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.0 2021-02-20T07:54:28.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2021-02-20T07:54:28.928 INFO:teuthology.orchestra.run.smithi106.stdout:34359738370 2021-02-20T07:54:28.943 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.1 2021-02-20T07:54:28.943 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2021-02-20T07:54:29.237 INFO:teuthology.orchestra.run.smithi106.stdout:34359738371 2021-02-20T07:54:29.252 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.2 2021-02-20T07:54:29.252 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2021-02-20T07:54:29.253 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T07:54:29.549 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:29.551 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2021-02-20T07:54:29.563 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T07:54:29.252657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.531930+0000","last_change":"2021-02-20T07:54:24.530975+0000","last_active":"2021-02-20T07:54:24.531930+0000","last_peered":"2021-02-20T07:54:24.531930+0000","last_clean":"2021-02-20T07:54:24.531930+0000","last_became_active":"2021-02-20T07:54:24.530653+0000","last_became_peered":"2021-02-20T07:54:24.530653+0000","last_unstale":"2021-02-20T07:54:24.531930+0000","last_undegraded":"2021-02-20T07:54:24.531930+0000","last_fullsized":"2021-02-20T07:54:24.531930+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.531895+0000","last_change":"2021-02-20T07:54:24.530766+0000","last_active":"2021-02-20T07:54:24.531895+0000","last_peered":"2021-02-20T07:54:24.531895+0000","last_clean":"2021-02-20T07:54:24.531895+0000","last_became_active":"2021-02-20T07:54:24.530439+0000","last_became_peered":"2021-02-20T07:54:24.530439+0000","last_unstale":"2021-02-20T07:54:24.531895+0000","last_undegraded":"2021-02-20T07:54:24.531895+0000","last_fullsized":"2021-02-20T07:54:24.531895+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529249+0000","last_change":"2021-02-20T07:54:24.524206+0000","last_active":"2021-02-20T07:54:24.529249+0000","last_peered":"2021-02-20T07:54:24.529249+0000","last_clean":"2021-02-20T07:54:24.529249+0000","last_became_active":"2021-02-20T07:54:24.523768+0000","last_became_peered":"2021-02-20T07:54:24.523768+0000","last_unstale":"2021-02-20T07:54:24.529249+0000","last_undegraded":"2021-02-20T07:54:24.529249+0000","last_fullsized":"2021-02-20T07:54:24.529249+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.525917+0000","last_change":"2021-02-20T07:54:24.524408+0000","last_active":"2021-02-20T07:54:24.525917+0000","last_peered":"2021-02-20T07:54:24.525917+0000","last_clean":"2021-02-20T07:54:24.525917+0000","last_became_active":"2021-02-20T07:54:24.524065+0000","last_became_peered":"2021-02-20T07:54:24.524065+0000","last_unstale":"2021-02-20T07:54:24.525917+0000","last_undegraded":"2021-02-20T07:54:24.525917+0000","last_fullsized":"2021-02-20T07:54:24.525917+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.528816+0000","last_change":"2021-02-20T07:54:24.523378+0000","last_active":"2021-02-20T07:54:24.528816+0000","last_peered":"2021-02-20T07:54:24.528816+0000","last_clean":"2021-02-20T07:54:24.528816+0000","last_became_active":"2021-02-20T07:54:24.523040+0000","last_became_peered":"2021-02-20T07:54:24.523040+0000","last_unstale":"2021-02-20T07:54:24.528816+0000","last_undegraded":"2021-02-20T07:54:24.528816+0000","last_fullsized":"2021-02-20T07:54:24.528816+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529158+0000","last_change":"2021-02-20T07:54:24.523852+0000","last_active":"2021-02-20T07:54:24.529158+0000","last_peered":"2021-02-20T07:54:24.529158+0000","last_clean":"2021-02-20T07:54:24.529158+0000","last_became_active":"2021-02-20T07:54:24.523471+0000","last_became_peered":"2021-02-20T07:54:24.523471+0000","last_unstale":"2021-02-20T07:54:24.529158+0000","last_undegraded":"2021-02-20T07:54:24.529158+0000","last_fullsized":"2021-02-20T07:54:24.529158+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.525924+0000","last_change":"2021-02-20T07:54:24.524284+0000","last_active":"2021-02-20T07:54:24.525924+0000","last_peered":"2021-02-20T07:54:24.525924+0000","last_clean":"2021-02-20T07:54:24.525924+0000","last_became_active":"2021-02-20T07:54:24.523913+0000","last_became_peered":"2021-02-20T07:54:24.523913+0000","last_unstale":"2021-02-20T07:54:24.525924+0000","last_undegraded":"2021-02-20T07:54:24.525924+0000","last_fullsized":"2021-02-20T07:54:24.525924+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.527444+0000","last_change":"2021-02-20T07:54:24.525232+0000","last_active":"2021-02-20T07:54:24.527444+0000","last_peered":"2021-02-20T07:54:24.527444+0000","last_clean":"2021-02-20T07:54:24.527444+0000","last_became_active":"2021-02-20T07:54:24.524858+0000","last_became_peered":"2021-02-20T07:54:24.524858+0000","last_unstale":"2021-02-20T07:54:24.527444+0000","last_undegraded":"2021-02-20T07:54:24.527444+0000","last_fullsized":"2021-02-20T07:54:24.527444+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529354+0000","last_change":"2021-02-20T07:54:24.525418+0000","last_active":"2021-02-20T07:54:24.529354+0000","last_peered":"2021-02-20T07:54:24.529354+0000","last_clean":"2021-02-20T07:54:24.529354+0000","last_became_active":"2021-02-20T07:54:24.524577+0000","last_became_peered":"2021-02-20T07:54:24.524577+0000","last_unstale":"2021-02-20T07:54:24.529354+0000","last_undegraded":"2021-02-20T07:54:24.529354+0000","last_fullsized":"2021-02-20T07:54:24.529354+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]}]}} 2021-02-20T07:54:29.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-02-20T07:54:29.862 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:29.865 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2021-02-20T07:54:29.876 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T07:54:29.252657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.531930+0000","last_change":"2021-02-20T07:54:24.530975+0000","last_active":"2021-02-20T07:54:24.531930+0000","last_peered":"2021-02-20T07:54:24.531930+0000","last_clean":"2021-02-20T07:54:24.531930+0000","last_became_active":"2021-02-20T07:54:24.530653+0000","last_became_peered":"2021-02-20T07:54:24.530653+0000","last_unstale":"2021-02-20T07:54:24.531930+0000","last_undegraded":"2021-02-20T07:54:24.531930+0000","last_fullsized":"2021-02-20T07:54:24.531930+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.531895+0000","last_change":"2021-02-20T07:54:24.530766+0000","last_active":"2021-02-20T07:54:24.531895+0000","last_peered":"2021-02-20T07:54:24.531895+0000","last_clean":"2021-02-20T07:54:24.531895+0000","last_became_active":"2021-02-20T07:54:24.530439+0000","last_became_peered":"2021-02-20T07:54:24.530439+0000","last_unstale":"2021-02-20T07:54:24.531895+0000","last_undegraded":"2021-02-20T07:54:24.531895+0000","last_fullsized":"2021-02-20T07:54:24.531895+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529249+0000","last_change":"2021-02-20T07:54:24.524206+0000","last_active":"2021-02-20T07:54:24.529249+0000","last_peered":"2021-02-20T07:54:24.529249+0000","last_clean":"2021-02-20T07:54:24.529249+0000","last_became_active":"2021-02-20T07:54:24.523768+0000","last_became_peered":"2021-02-20T07:54:24.523768+0000","last_unstale":"2021-02-20T07:54:24.529249+0000","last_undegraded":"2021-02-20T07:54:24.529249+0000","last_fullsized":"2021-02-20T07:54:24.529249+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.525917+0000","last_change":"2021-02-20T07:54:24.524408+0000","last_active":"2021-02-20T07:54:24.525917+0000","last_peered":"2021-02-20T07:54:24.525917+0000","last_clean":"2021-02-20T07:54:24.525917+0000","last_became_active":"2021-02-20T07:54:24.524065+0000","last_became_peered":"2021-02-20T07:54:24.524065+0000","last_unstale":"2021-02-20T07:54:24.525917+0000","last_undegraded":"2021-02-20T07:54:24.525917+0000","last_fullsized":"2021-02-20T07:54:24.525917+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.528816+0000","last_change":"2021-02-20T07:54:24.523378+0000","last_active":"2021-02-20T07:54:24.528816+0000","last_peered":"2021-02-20T07:54:24.528816+0000","last_clean":"2021-02-20T07:54:24.528816+0000","last_became_active":"2021-02-20T07:54:24.523040+0000","last_became_peered":"2021-02-20T07:54:24.523040+0000","last_unstale":"2021-02-20T07:54:24.528816+0000","last_undegraded":"2021-02-20T07:54:24.528816+0000","last_fullsized":"2021-02-20T07:54:24.528816+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529158+0000","last_change":"2021-02-20T07:54:24.523852+0000","last_active":"2021-02-20T07:54:24.529158+0000","last_peered":"2021-02-20T07:54:24.529158+0000","last_clean":"2021-02-20T07:54:24.529158+0000","last_became_active":"2021-02-20T07:54:24.523471+0000","last_became_peered":"2021-02-20T07:54:24.523471+0000","last_unstale":"2021-02-20T07:54:24.529158+0000","last_undegraded":"2021-02-20T07:54:24.529158+0000","last_fullsized":"2021-02-20T07:54:24.529158+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.525924+0000","last_change":"2021-02-20T07:54:24.524284+0000","last_active":"2021-02-20T07:54:24.525924+0000","last_peered":"2021-02-20T07:54:24.525924+0000","last_clean":"2021-02-20T07:54:24.525924+0000","last_became_active":"2021-02-20T07:54:24.523913+0000","last_became_peered":"2021-02-20T07:54:24.523913+0000","last_unstale":"2021-02-20T07:54:24.525924+0000","last_undegraded":"2021-02-20T07:54:24.525924+0000","last_fullsized":"2021-02-20T07:54:24.525924+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.527444+0000","last_change":"2021-02-20T07:54:24.525232+0000","last_active":"2021-02-20T07:54:24.527444+0000","last_peered":"2021-02-20T07:54:24.527444+0000","last_clean":"2021-02-20T07:54:24.527444+0000","last_became_active":"2021-02-20T07:54:24.524858+0000","last_became_peered":"2021-02-20T07:54:24.524858+0000","last_unstale":"2021-02-20T07:54:24.527444+0000","last_undegraded":"2021-02-20T07:54:24.527444+0000","last_fullsized":"2021-02-20T07:54:24.527444+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:24.529354+0000","last_change":"2021-02-20T07:54:24.525418+0000","last_active":"2021-02-20T07:54:24.529354+0000","last_peered":"2021-02-20T07:54:24.529354+0000","last_clean":"2021-02-20T07:54:24.529354+0000","last_became_active":"2021-02-20T07:54:24.524577+0000","last_became_peered":"2021-02-20T07:54:24.524577+0000","last_unstale":"2021-02-20T07:54:24.529354+0000","last_undegraded":"2021-02-20T07:54:24.529354+0000","last_fullsized":"2021-02-20T07:54:24.529354+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:23.494278+0000","last_clean_scrub_stamp":"2021-02-20T07:54:23.494278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]}]}} 2021-02-20T07:54:29.877 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2021-02-20T07:54:29.878 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2021-02-20T07:54:29.878 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2021-02-20T07:54:29.878 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2021-02-20T07:54:30.237 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:30.237 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T07:54:30.251 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2021-02-20T07:54:30.252 INFO:teuthology.run_tasks:Running task ceph... 2021-02-20T07:54:30.264 INFO:tasks.ceph:Creating ceph cluster cluster2... 2021-02-20T07:54:30.264 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-02-20T07:54:30.265 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi/5897891', 'branch': 'wip-mgolub-testing', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'email': 'to.my.trociny@gmail.com', 'first_in_suite': False, 'job_id': '5897891', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'trociny-2021-02-20_07:16:52-rbd-wip-mgolub-testing-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-mgolub-testing'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mgolub-testing', 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, 'owner': 'scheduled_trociny@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-mgolub-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d318d42d502ea13c78f2af2499efd19dcc7e174b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b', 'targets': {'smithi106.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1mnk0O6+NreOssSFrZ95wpiF8tx9IRDQM3nDni9DDXQbHBttgEFNAoVR5+7i1QGU1IjrmGogPN1ot03XpA9AbaT/KvymYBq2iL5O5eTwSJ6OAPai1vNH2KD91ISBja7+c9LmcoTu69zflvewYGWRD2Av8vnC3JEGqC/c1ROTnFaxOAmIz8OTxmN7XxxRmq/D2QOBvFFdV1MsshGIPmFGWj7MgYSDZoe7TWpjXJo3L1HG9fa79HoOFSrsn/FvBTQ772Y0YluDkmbNsGXUuHB09bchr1XBIQNqbnlZv/Mx2PschkZIBUc2Ob5d3jpWNjvjJVt7pzVutl4PGvip1eWlL', 'smithi114.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDstIYH2/PVs6ZJhqo/CwHh6FWOoxqMGz3sP3ZIantQz95ZbsZA0mJ/1DV7UlYIghJhw3XsASkRnCEPgYH0yPIU7PeG4IyWjvpEbKNNUXjUmPsEENYZL4IIIY6xFkuKZZpWRvS1WKebQmAPuGWyQAXS2E+Xh0WGQIcy2kC2pPEICjmpjjH8dvtN9+i3xDcbcUZ7T/EnJh+KeCtTPQEFs2kdZCP4m4DMG0hiwqlwPWWgnA2QXMLguc4v19RWDzot2lhDXNyRcq9PGghJifN+R5UbgKRssiEd4UxM44CL1MSdo89fSy04L0VpoHgnzeUn9xhYFodHp1p0EPC9laBn5rv/'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd318d42d502ea13c78f2af2499efd19dcc7e174b'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,object-map,fast-diff', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.29353'} 2021-02-20T07:54:30.265 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T07:54:30.268 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-02-20T07:54:30.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T07:54:30.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2021-02-20T07:54:30.333 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:30.333 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2021-02-20T07:54:30.381 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-02-20T07:54:30.382 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-02-20T07:54:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-02-20T07:54:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:54:30.431 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 342 Links: 1 2021-02-20T07:54:30.431 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:54:30.431 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2021-02-20 07:51:19.649812077 +0000 2021-02-20T07:54:30.431 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2021-02-20 07:51:19.557812079 +0000 2021-02-20T07:54:30.432 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2021-02-20 07:51:19.557812079 +0000 2021-02-20T07:54:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2021-02-20T07:54:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2021-02-20T07:54:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2021-02-20T07:54:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00038366 s, 1.3 MB/s 2021-02-20T07:54:30.446 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-02-20T07:54:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-02-20T07:54:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:54:30.495 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 348 Links: 1 2021-02-20T07:54:30.495 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:54:30.496 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2021-02-20 07:51:19.801812074 +0000 2021-02-20T07:54:30.496 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2021-02-20 07:51:19.797812074 +0000 2021-02-20T07:54:30.496 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2021-02-20 07:51:19.797812074 +0000 2021-02-20T07:54:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2021-02-20T07:54:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2021-02-20T07:54:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2021-02-20T07:54:30.505 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000335361 s, 1.5 MB/s 2021-02-20T07:54:30.509 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-02-20T07:54:30.558 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-02-20T07:54:30.559 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:54:30.559 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 354 Links: 1 2021-02-20T07:54:30.560 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:54:30.560 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2021-02-20 07:51:20.005812069 +0000 2021-02-20T07:54:30.560 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2021-02-20 07:51:20.001812069 +0000 2021-02-20T07:54:30.560 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2021-02-20 07:51:20.001812069 +0000 2021-02-20T07:54:30.561 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2021-02-20T07:54:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2021-02-20T07:54:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2021-02-20T07:54:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000354943 s, 1.4 MB/s 2021-02-20T07:54:30.573 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-02-20T07:54:30.623 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-02-20T07:54:30.623 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-02-20T07:54:30.624 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 361 Links: 1 2021-02-20T07:54:30.624 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-02-20T07:54:30.624 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2021-02-20 07:51:20.369812061 +0000 2021-02-20T07:54:30.625 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2021-02-20 07:51:20.249812064 +0000 2021-02-20T07:54:30.625 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2021-02-20 07:51:20.249812064 +0000 2021-02-20T07:54:30.625 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2021-02-20T07:54:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2021-02-20T07:54:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2021-02-20T07:54:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00043724 s, 1.2 MB/s 2021-02-20T07:54:30.636 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:30.637 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T07:54:30.637 INFO:tasks.ceph:Generating config... 2021-02-20T07:54:30.639 INFO:tasks.ceph:[client] debug rbd = 20 2021-02-20T07:54:30.640 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-02-20T07:54:30.640 INFO:tasks.ceph:[client] log to stderr = False 2021-02-20T07:54:30.640 INFO:tasks.ceph:[client] rbd default features = 125 2021-02-20T07:54:30.641 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2021-02-20T07:54:30.641 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:30.641 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:30.641 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:30.642 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:30.642 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:30.642 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:30.643 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:30.643 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:30.643 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-02-20T07:54:30.644 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-02-20T07:54:30.644 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-02-20T07:54:30.644 INFO:tasks.ceph:[mgr] debug ms = 1 2021-02-20T07:54:30.645 INFO:tasks.ceph:[mon] debug mon = 20 2021-02-20T07:54:30.645 INFO:tasks.ceph:[mon] debug ms = 1 2021-02-20T07:54:30.645 INFO:tasks.ceph:[mon] debug paxos = 20 2021-02-20T07:54:30.645 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-02-20T07:54:30.646 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2021-02-20T07:54:30.646 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-02-20T07:54:30.646 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-02-20T07:54:30.647 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-02-20T07:54:30.647 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-02-20T07:54:30.647 INFO:tasks.ceph:[osd] debug filestore = 20 2021-02-20T07:54:30.648 INFO:tasks.ceph:[osd] debug journal = 20 2021-02-20T07:54:30.648 INFO:tasks.ceph:[osd] debug ms = 20 2021-02-20T07:54:30.648 INFO:tasks.ceph:[osd] debug osd = 25 2021-02-20T07:54:30.648 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-02-20T07:54:30.649 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-02-20T07:54:30.649 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-02-20T07:54:30.649 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-02-20T07:54:30.650 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-02-20T07:54:30.650 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-02-20T07:54:30.650 INFO:tasks.ceph:Setting up cluster2.mon.a... 2021-02-20T07:54:30.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2021-02-20T07:54:30.711 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.keyring 2021-02-20T07:54:30.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2021-02-20T07:54:30.792 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T07:54:30.850 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.114')] 2021-02-20T07:54:30.851 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.114', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-02-20T07:54:30.853 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:30.854 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T07:54:30.898 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.114 --print /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:30.970 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:30.971 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: generated fsid d75c5c24-a5ff-4862-8e7e-70367438e25f 2021-02-20T07:54:30.971 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 0 2021-02-20T07:54:30.971 INFO:teuthology.orchestra.run.smithi114.stdout:fsid d75c5c24-a5ff-4862-8e7e-70367438e25f 2021-02-20T07:54:30.971 INFO:teuthology.orchestra.run.smithi114.stdout:last_changed 2021-02-20T07:54:30.973176+0000 2021-02-20T07:54:30.971 INFO:teuthology.orchestra.run.smithi114.stdout:created 2021-02-20T07:54:30.973176+0000 2021-02-20T07:54:30.972 INFO:teuthology.orchestra.run.smithi114.stdout:min_mon_release 0 (unknown) 2021-02-20T07:54:30.972 INFO:teuthology.orchestra.run.smithi114.stdout:election_strategy: 1 2021-02-20T07:54:30.972 INFO:teuthology.orchestra.run.smithi114.stdout:0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2021-02-20T07:54:30.973 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2021-02-20T07:54:30.974 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-02-20T07:54:31.021 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID d75c5c24-a5ff-4862-8e7e-70367438e25f... 2021-02-20T07:54:31.022 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T07:54:31.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-02-20T07:54:31.066 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2021-02-20T07:54:31.067 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2021-02-20T07:54:31.067 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T07:54:31.067 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2021-02-20T07:54:31.067 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.068 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2021-02-20T07:54:31.068 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.068 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2021-02-20T07:54:31.068 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.068 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2021-02-20T07:54:31.069 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2021-02-20T07:54:31.069 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.069 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2021-02-20T07:54:31.069 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2021-02-20T07:54:31.069 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.070 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T07:54:31.070 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2021-02-20T07:54:31.070 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.070 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T07:54:31.071 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T07:54:31.071 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.071 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2021-02-20T07:54:31.071 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.071 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2021-02-20T07:54:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2021-02-20T07:54:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2021-02-20T07:54:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2021-02-20T07:54:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2021-02-20T07:54:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2021-02-20T07:54:31.073 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2021-02-20T07:54:31.073 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T07:54:31.073 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T07:54:31.073 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2021-02-20T07:54:31.073 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.074 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T07:54:31.074 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.074 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2021-02-20T07:54:31.074 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.075 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2021-02-20T07:54:31.075 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.075 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2021-02-20T07:54:31.075 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2021-02-20T07:54:31.075 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2021-02-20T07:54:31.076 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = 172.21.15.114 2021-02-20T07:54:31.076 INFO:teuthology.orchestra.run.smithi106.stdout: mon client directed command retry = 5 2021-02-20T07:54:31.076 INFO:teuthology.orchestra.run.smithi106.stdout: ms inject socket failures = 5000 2021-02-20T07:54:31.076 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d75c5c24-a5ff-4862-8e7e-70367438e25f 2021-02-20T07:54:31.076 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.077 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2021-02-20T07:54:31.077 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2021-02-20T07:54:31.077 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.077 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2021-02-20T07:54:31.077 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2021-02-20T07:54:31.078 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.078 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2021-02-20T07:54:31.078 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2021-02-20T07:54:31.078 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.078 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2021-02-20T07:54:31.079 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2021-02-20T07:54:31.079 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2021-02-20T07:54:31.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.079 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2021-02-20T07:54:31.079 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2021-02-20T07:54:31.080 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.080 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2021-02-20T07:54:31.080 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.080 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2021-02-20T07:54:31.080 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.081 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2021-02-20T07:54:31.081 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.081 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2021-02-20T07:54:31.081 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2021-02-20T07:54:31.082 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.082 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2021-02-20T07:54:31.082 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2021-02-20T07:54:31.082 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2021-02-20T07:54:31.082 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore compression algorithm = snappy 2021-02-20T07:54:31.083 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore compression mode = aggressive 2021-02-20T07:54:31.083 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2021-02-20T07:54:31.083 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 20 2021-02-20T07:54:31.083 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 20 2021-02-20T07:54:31.083 INFO:teuthology.orchestra.run.smithi106.stdout: debug filestore = 20 2021-02-20T07:54:31.084 INFO:teuthology.orchestra.run.smithi106.stdout: debug journal = 20 2021-02-20T07:54:31.084 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 20 2021-02-20T07:54:31.084 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 25 2021-02-20T07:54:31.084 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 10 2021-02-20T07:54:31.084 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T07:54:31.085 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2021-02-20T07:54:31.085 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2021-02-20T07:54:31.085 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2021-02-20T07:54:31.085 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2021-02-20T07:54:31.085 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.086 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2021-02-20T07:54:31.086 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2021-02-20T07:54:31.086 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2021-02-20T07:54:31.086 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2021-02-20T07:54:31.087 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2021-02-20T07:54:31.087 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:31.087 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:31.087 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2021-02-20T07:54:31.087 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.088 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2021-02-20T07:54:31.088 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2021-02-20T07:54:31.088 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2021-02-20T07:54:31.088 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2021-02-20T07:54:31.088 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2021-02-20T07:54:31.089 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2021-02-20T07:54:31.089 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2021-02-20T07:54:31.089 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:31.089 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2021-02-20T07:54:31.089 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2021-02-20T07:54:31.090 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:31.090 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-02-20T07:54:31.090 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2021-02-20T07:54:31.090 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2021-02-20T07:54:31.090 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2021-02-20T07:54:31.091 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2021-02-20T07:54:31.091 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T07:54:31.091 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T07:54:31.091 INFO:teuthology.orchestra.run.smithi106.stdout: debug rbd = 20 2021-02-20T07:54:31.092 INFO:teuthology.orchestra.run.smithi106.stdout: debug rbd_mirror = 15 2021-02-20T07:54:31.092 INFO:teuthology.orchestra.run.smithi106.stdout: log to stderr = False 2021-02-20T07:54:31.092 INFO:teuthology.orchestra.run.smithi106.stdout: rbd default features = 125 2021-02-20T07:54:31.092 INFO:teuthology.orchestra.run.smithi106.stdout: rbd mirror image policy type = simple 2021-02-20T07:54:31.092 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2021-02-20T07:54:31.093 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.0] 2021-02-20T07:54:31.093 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.093 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.093 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.1] 2021-02-20T07:54:31.093 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.094 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.094 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.2] 2021-02-20T07:54:31.094 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.094 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.095 INFO:teuthology.orchestra.run.smithi106.stdout:[client.mirror.3] 2021-02-20T07:54:31.095 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.095 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.097 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2021-02-20T07:54:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2021-02-20T07:54:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-02-20T07:54:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2021-02-20T07:54:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2021-02-20T07:54:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2021-02-20T07:54:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2021-02-20T07:54:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2021-02-20T07:54:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2021-02-20T07:54:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2021-02-20T07:54:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2021-02-20T07:54:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2021-02-20T07:54:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2021-02-20T07:54:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2021-02-20T07:54:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2021-02-20T07:54:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2021-02-20T07:54:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2021-02-20T07:54:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2021-02-20T07:54:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2021-02-20T07:54:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2021-02-20T07:54:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2021-02-20T07:54:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2021-02-20T07:54:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-02-20T07:54:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2021-02-20T07:54:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2021-02-20T07:54:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-02-20T07:54:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2021-02-20T07:54:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2021-02-20T07:54:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2021-02-20T07:54:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2021-02-20T07:54:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2021-02-20T07:54:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = 172.21.15.114 2021-02-20T07:54:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2021-02-20T07:54:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 5000 2021-02-20T07:54:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = d75c5c24-a5ff-4862-8e7e-70367438e25f 2021-02-20T07:54:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.107 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2021-02-20T07:54:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2021-02-20T07:54:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2021-02-20T07:54:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2021-02-20T07:54:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2021-02-20T07:54:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2021-02-20T07:54:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2021-02-20T07:54:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2021-02-20T07:54:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2021-02-20T07:54:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2021-02-20T07:54:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2021-02-20T07:54:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2021-02-20T07:54:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2021-02-20T07:54:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2021-02-20T07:54:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2021-02-20T07:54:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2021-02-20T07:54:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2021-02-20T07:54:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2021-02-20T07:54:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2021-02-20T07:54:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression algorithm = snappy 2021-02-20T07:54:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression mode = aggressive 2021-02-20T07:54:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2021-02-20T07:54:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2021-02-20T07:54:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2021-02-20T07:54:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: debug filestore = 20 2021-02-20T07:54:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: debug journal = 20 2021-02-20T07:54:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 20 2021-02-20T07:54:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 25 2021-02-20T07:54:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2021-02-20T07:54:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2021-02-20T07:54:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2021-02-20T07:54:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2021-02-20T07:54:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2021-02-20T07:54:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2021-02-20T07:54:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.117 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2021-02-20T07:54:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2021-02-20T07:54:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2021-02-20T07:54:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2021-02-20T07:54:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2021-02-20T07:54:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2021-02-20T07:54:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.119 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2021-02-20T07:54:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2021-02-20T07:54:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2021-02-20T07:54:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2021-02-20T07:54:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2021-02-20T07:54:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2021-02-20T07:54:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2021-02-20T07:54:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2021-02-20T07:54:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2021-02-20T07:54:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2021-02-20T07:54:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2021-02-20T07:54:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:31.121 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2021-02-20T07:54:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2021-02-20T07:54:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2021-02-20T07:54:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2021-02-20T07:54:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-02-20T07:54:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-02-20T07:54:31.123 INFO:teuthology.orchestra.run.smithi114.stdout: debug rbd = 20 2021-02-20T07:54:31.123 INFO:teuthology.orchestra.run.smithi114.stdout: debug rbd_mirror = 15 2021-02-20T07:54:31.123 INFO:teuthology.orchestra.run.smithi114.stdout: log to stderr = False 2021-02-20T07:54:31.123 INFO:teuthology.orchestra.run.smithi114.stdout: rbd default features = 125 2021-02-20T07:54:31.123 INFO:teuthology.orchestra.run.smithi114.stdout: rbd mirror image policy type = simple 2021-02-20T07:54:31.124 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2021-02-20T07:54:31.124 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.0] 2021-02-20T07:54:31.124 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.124 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.125 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.1] 2021-02-20T07:54:31.125 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.125 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.125 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.2] 2021-02-20T07:54:31.125 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.126 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.126 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.3] 2021-02-20T07:54:31.126 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-02-20T07:54:31.126 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-02-20T07:54:31.127 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2021-02-20T07:54:31.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2021-02-20T07:54:31.218 INFO:tasks.ceph:Copying monmap to all nodes... 2021-02-20T07:54:31.218 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:31.219 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2021-02-20T07:54:31.266 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:31.267 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2021-02-20T07:54:31.313 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2021-02-20T07:54:31.314 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:31.314 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T07:54:31.314 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T07:54:31.340 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:31.341 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:31.389 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:54:31.390 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:31.390 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/cluster2.keyring 2021-02-20T07:54:31.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-02-20T07:54:31.416 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:31.417 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:31.465 INFO:tasks.ceph:Setting up mon nodes... 2021-02-20T07:54:31.466 INFO:tasks.ceph:Setting up mgr nodes... 2021-02-20T07:54:31.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T07:54:31.508 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2021-02-20T07:54:31.510 INFO:tasks.ceph:Setting up mds nodes... 2021-02-20T07:54:31.511 INFO:tasks.ceph_client:Setting up client nodes... 2021-02-20T07:54:31.511 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2021-02-20T07:54:31.585 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/cluster2.client.0.keyring 2021-02-20T07:54:31.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T07:54:31.632 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2021-02-20T07:54:31.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T07:54:31.721 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2021-02-20T07:54:31.733 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T07:54:31.808 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2021-02-20T07:54:31.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T07:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2021-02-20T07:54:31.909 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T07:54:31.983 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2021-02-20T07:54:31.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T07:54:32.074 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2021-02-20T07:54:32.084 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T07:54:32.161 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2021-02-20T07:54:32.173 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T07:54:32.248 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2021-02-20T07:54:32.260 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-02-20T07:54:32.260 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-02-20T07:54:32.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2021-02-20T07:54:32.318 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:32.318 INFO:tasks.ceph:role: cluster2.osd.0 2021-02-20T07:54:32.319 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:54:32.319 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-02-20T07:54:32.833 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:32.834 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:32.834 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:32.834 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:32.834 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:32.835 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:32.835 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:32.835 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:32.835 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:32.836 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2021-02-20T07:54:32.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2021-02-20T07:54:32.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2021-02-20T07:54:32.926 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:32.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:32.928 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2021-02-20T07:54:32.986 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:32.987 INFO:tasks.ceph:role: cluster2.osd.1 2021-02-20T07:54:32.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:54:32.987 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-02-20T07:54:33.492 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:33.492 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:33.492 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:33.493 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:33.493 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:33.493 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:33.493 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:33.494 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:33.494 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:33.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2021-02-20T07:54:33.495 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2021-02-20T07:54:33.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2021-02-20T07:54:33.573 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:33.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:33.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2021-02-20T07:54:33.630 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-02-20T07:54:33.630 INFO:tasks.ceph:role: cluster2.osd.2 2021-02-20T07:54:33.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:54:33.631 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-02-20T07:54:34.139 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-02-20T07:54:34.139 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2021-02-20T07:54:34.140 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-02-20T07:54:34.140 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-02-20T07:54:34.140 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2021-02-20T07:54:34.140 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-02-20T07:54:34.141 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-02-20T07:54:34.141 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-02-20T07:54:34.141 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-02-20T07:54:34.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2021-02-20T07:54:34.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2021-02-20T07:54:34.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2021-02-20T07:54:34.227 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2021-02-20T07:54:34.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:54:34.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.314+0000 7f8f148a2080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2021-02-20T07:54:34.316 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.318+0000 7f8f148a2080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2021-02-20T07:54:34.316 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.318+0000 7f8f148a2080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T07:54:34.316 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.318+0000 7f8f148a2080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T07:54:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.318+0000 7f8f148a2080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-02-20T07:54:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:34.318+0000 7f8f148a2080 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2021-02-20T07:54:37.280 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2021-02-20T07:54:37.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2021-02-20T07:54:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2021-02-20T07:54:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T07:54:37.419 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T07:54:37.419 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-02-20T07:54:37.419 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:37.418+0000 7f119ab8b080 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2021-02-20T07:54:40.396 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2021-02-20T07:54:40.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:40.533 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2021-02-20T07:54:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2021-02-20T07:54:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T07:54:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T07:54:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-02-20T07:54:40.535 INFO:teuthology.orchestra.run.smithi114.stderr:2021-02-20T07:54:40.534+0000 7fd8e93eb080 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2021-02-20T07:54:43.483 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2021-02-20T07:54:43.543 INFO:tasks.ceph:Reading keys from all nodes... 2021-02-20T07:54:43.543 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:43.544 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2021-02-20T07:54:43.559 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2021-02-20T07:54:43.597 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2021-02-20T07:54:43.654 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2021-02-20T07:54:43.710 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2021-02-20T07:54:43.710 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2021-02-20T07:54:43.717 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.718 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2021-02-20T07:54:43.757 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.757 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2021-02-20T07:54:43.805 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.806 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2021-02-20T07:54:43.853 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.853 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2021-02-20T07:54:43.903 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.903 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2021-02-20T07:54:43.949 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.950 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2021-02-20T07:54:43.997 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:43.998 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2021-02-20T07:54:44.045 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:44.045 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2021-02-20T07:54:44.094 INFO:tasks.ceph:Adding keys to all mons... 2021-02-20T07:54:44.094 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/cluster2.keyring 2021-02-20T07:54:44.148 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2021-02-20T07:54:44.149 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgrsSGHhAAQEGEZjAfCStMDbfBIQ8TQQ== 2021-02-20T07:54:44.149 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2021-02-20T07:54:44.149 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA6wDBgx976EhAArhNuwFilUVGtbGlcfkFr6A== 2021-02-20T07:54:44.150 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2021-02-20T07:54:44.150 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA9wDBgoxsVGRAAIYUjFOBL95rEEQXxT3/EPg== 2021-02-20T07:54:44.150 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2021-02-20T07:54:44.150 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBAwDBgcQD7HxAAbmv4gFFeENjFlarhDNDGKw== 2021-02-20T07:54:44.150 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2021-02-20T07:54:44.151 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBg5J8YIxAAaJ5QqZLS1iyb9Oc2wDJfJA== 2021-02-20T07:54:44.151 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror] 2021-02-20T07:54:44.151 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgsv7iJRAA66l+BtFXhzimNRvLty2oag== 2021-02-20T07:54:44.151 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.0] 2021-02-20T07:54:44.152 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgcZ4oKxAAIFAE7LeH+2jbzKoLqwGq2w== 2021-02-20T07:54:44.152 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.1] 2021-02-20T07:54:44.152 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgySZfMBAAzh626EkvxS4/VfIFwl8ong== 2021-02-20T07:54:44.152 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.2] 2021-02-20T07:54:44.152 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgaSqfNRAAKpc7qJH366V7kECwz8I3Xw== 2021-02-20T07:54:44.153 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.3] 2021-02-20T07:54:44.153 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA3wDBgPDLTOhAAq9tkybsIX/gbd907Cj3Y1g== 2021-02-20T07:54:44.153 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.4] 2021-02-20T07:54:44.153 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA4wDBgjBGLBBAA7nKrc/vDnz0fymfMVz8MkA== 2021-02-20T07:54:44.154 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.5] 2021-02-20T07:54:44.154 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA4wDBg/nTUCRAAKzxR8QOFOzFDY0pahwx6sw== 2021-02-20T07:54:44.154 INFO:teuthology.orchestra.run.smithi114.stdout:[client.mirror.6] 2021-02-20T07:54:44.154 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQA4wDBgAkoBDxAA6hJZarSz0bKecgarZK8JNA== 2021-02-20T07:54:44.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-02-20T07:54:44.241 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:44.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:44.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-02-20T07:54:44.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.720 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:44.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:45.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:45.064 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-02-20T07:54:45.156 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-02-20T07:54:45.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2021-02-20T07:54:45.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2021-02-20T07:54:45.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2021-02-20T07:54:45.328 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2021-02-20T07:54:45.377 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2021-02-20T07:54:45.377 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2021-02-20T07:54:45.378 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2021-02-20T07:54:45.421 INFO:tasks.ceph.cluster2.mon.a:Started 2021-02-20T07:54:45.422 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2021-02-20T07:54:45.422 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2021-02-20T07:54:45.423 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2021-02-20T07:54:45.425 INFO:tasks.ceph.cluster2.mgr.x:Started 2021-02-20T07:54:45.425 INFO:tasks.ceph:Setting crush tunables to default 2021-02-20T07:54:45.426 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd crush tunables default 2021-02-20T07:54:45.767 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2021-02-20T07:54:45.778 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2021-02-20T07:54:45.779 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:45.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2021-02-20T07:54:45.793 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:45.794 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2021-02-20T07:54:45.849 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:54:45.849 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2021-02-20T07:54:45.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd new 6a4ba8fc-9774-4985-a8fd-d8748ef5827d 0 2021-02-20T07:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout:0 2021-02-20T07:54:46.185 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd new 2597dc5f-c106-482a-88f0-5cd9bda3b396 1 2021-02-20T07:54:46.396 INFO:teuthology.orchestra.run.smithi114.stdout:1 2021-02-20T07:54:46.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd new f5b114e5-2b5f-41ad-ba17-4749ce787a24 2 2021-02-20T07:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout:2 2021-02-20T07:54:46.645 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2021-02-20T07:54:46.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2021-02-20T07:54:46.648 INFO:tasks.ceph.cluster2.osd.0:Started 2021-02-20T07:54:46.648 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2021-02-20T07:54:46.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2021-02-20T07:54:46.650 INFO:tasks.ceph.cluster2.osd.1:Started 2021-02-20T07:54:46.651 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2021-02-20T07:54:46.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2021-02-20T07:54:46.655 INFO:tasks.ceph.cluster2.osd.2:Started 2021-02-20T07:54:46.656 INFO:tasks.ceph:Waiting for OSDs to come up 2021-02-20T07:54:46.959 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T07:54:47.159 INFO:teuthology.misc.health.smithi114.stdout: 2021-02-20T07:54:47.160 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":5,"fsid":"d75c5c24-a5ff-4862-8e7e-70367438e25f","created":"2021-02-20T07:54:45.550274+0000","modified":"2021-02-20T07:54:46.632546+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6a4ba8fc-9774-4985-a8fd-d8748ef5827d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2597dc5f-c106-482a-88f0-5cd9bda3b396","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f5b114e5-2b5f-41ad-ba17-4749ce787a24","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:47.171 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-02-20T07:54:47.257 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T07:54:47.258+0000 7f8053115080 -1 Falling back to public interface 2021-02-20T07:54:47.265 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T07:54:47.266+0000 7fc28a81b080 -1 Falling back to public interface 2021-02-20T07:54:47.273 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:54:47.274+0000 7ff50f119080 -1 Falling back to public interface 2021-02-20T07:54:50.934 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T07:54:50.934+0000 7f8053115080 -1 osd.2 0 log_to_monitors {default=true} 2021-02-20T07:54:51.012 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:54:51.014+0000 7ff50f119080 -1 osd.0 0 log_to_monitors {default=true} 2021-02-20T07:54:51.045 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T07:54:51.046+0000 7fc28a81b080 -1 osd.1 0 log_to_monitors {default=true} 2021-02-20T07:54:52.595 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T07:54:52.594+0000 7f804a5de700 -1 osd.2 0 waiting for initial osdmap 2021-02-20T07:54:52.596 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:54:52.598+0000 7ff507de5700 -1 osd.0 0 waiting for initial osdmap 2021-02-20T07:54:52.597 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T07:54:52.598+0000 7fc281ce4700 -1 osd.1 0 waiting for initial osdmap 2021-02-20T07:54:53.474 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-02-20T07:54:53.827 INFO:teuthology.misc.health.smithi114.stdout: 2021-02-20T07:54:53.828 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":8,"fsid":"d75c5c24-a5ff-4862-8e7e-70367438e25f","created":"2021-02-20T07:54:45.550274+0000","modified":"2021-02-20T07:54:53.592939+0000","last_up_change":"2021-02-20T07:54:53.592939+0000","last_in_change":"2021-02-20T07:54:53.592939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6a4ba8fc-9774-4985-a8fd-d8748ef5827d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":15703},{"type":"v1","addr":"172.21.15.114:6817","nonce":15703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":15703},{"type":"v1","addr":"172.21.15.114:6819","nonce":15703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":15703},{"type":"v1","addr":"172.21.15.114:6823","nonce":15703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":15703},{"type":"v1","addr":"172.21.15.114:6821","nonce":15703}]},"public_addr":"172.21.15.114:6817/15703","cluster_addr":"172.21.15.114:6819/15703","heartbeat_back_addr":"172.21.15.114:6823/15703","heartbeat_front_addr":"172.21.15.114:6821/15703","state":["exists","up"]},{"osd":1,"uuid":"2597dc5f-c106-482a-88f0-5cd9bda3b396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":15704},{"type":"v1","addr":"172.21.15.114:6809","nonce":15704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":15704},{"type":"v1","addr":"172.21.15.114:6811","nonce":15704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":15704},{"type":"v1","addr":"172.21.15.114:6815","nonce":15704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":15704},{"type":"v1","addr":"172.21.15.114:6813","nonce":15704}]},"public_addr":"172.21.15.114:6809/15704","cluster_addr":"172.21.15.114:6811/15704","heartbeat_back_addr":"172.21.15.114:6815/15704","heartbeat_front_addr":"172.21.15.114:6813/15704","state":["exists","up"]},{"osd":2,"uuid":"f5b114e5-2b5f-41ad-ba17-4749ce787a24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":15705},{"type":"v1","addr":"172.21.15.114:6801","nonce":15705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":15705},{"type":"v1","addr":"172.21.15.114:6803","nonce":15705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":15705},{"type":"v1","addr":"172.21.15.114:6807","nonce":15705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":15705},{"type":"v1","addr":"172.21.15.114:6805","nonce":15705}]},"public_addr":"172.21.15.114:6801/15705","cluster_addr":"172.21.15.114:6803/15705","heartbeat_back_addr":"172.21.15.114:6807/15705","heartbeat_front_addr":"172.21.15.114:6805/15705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:53.840 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-02-20T07:54:53.841 INFO:tasks.ceph:Creating RBD pool 2021-02-20T07:54:53.842 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd pool create rbd 8 2021-02-20T07:54:54.619 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'rbd' created 2021-02-20T07:54:54.635 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-02-20T07:54:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:enabled application 'rbd' on pool 'rbd' 2021-02-20T07:54:55.645 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2021-02-20T07:54:55.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T07:54:55.648 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-02-20T07:54:55.944 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:55.944 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"fsid":"d75c5c24-a5ff-4862-8e7e-70367438e25f","created":"2021-02-20T07:54:45.550274+0000","modified":"2021-02-20T07:54:55.609901+0000","last_up_change":"2021-02-20T07:54:53.592939+0000","last_in_change":"2021-02-20T07:54:53.592939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:53.662980+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:54.212713+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6a4ba8fc-9774-4985-a8fd-d8748ef5827d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":15703},{"type":"v1","addr":"172.21.15.114:6817","nonce":15703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":15703},{"type":"v1","addr":"172.21.15.114:6819","nonce":15703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":15703},{"type":"v1","addr":"172.21.15.114:6823","nonce":15703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":15703},{"type":"v1","addr":"172.21.15.114:6821","nonce":15703}]},"public_addr":"172.21.15.114:6817/15703","cluster_addr":"172.21.15.114:6819/15703","heartbeat_back_addr":"172.21.15.114:6823/15703","heartbeat_front_addr":"172.21.15.114:6821/15703","state":["exists","up"]},{"osd":1,"uuid":"2597dc5f-c106-482a-88f0-5cd9bda3b396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":15704},{"type":"v1","addr":"172.21.15.114:6809","nonce":15704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":15704},{"type":"v1","addr":"172.21.15.114:6811","nonce":15704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":15704},{"type":"v1","addr":"172.21.15.114:6815","nonce":15704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":15704},{"type":"v1","addr":"172.21.15.114:6813","nonce":15704}]},"public_addr":"172.21.15.114:6809/15704","cluster_addr":"172.21.15.114:6811/15704","heartbeat_back_addr":"172.21.15.114:6815/15704","heartbeat_front_addr":"172.21.15.114:6813/15704","state":["exists","up"]},{"osd":2,"uuid":"f5b114e5-2b5f-41ad-ba17-4749ce787a24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":15705},{"type":"v1","addr":"172.21.15.114:6801","nonce":15705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":15705},{"type":"v1","addr":"172.21.15.114:6803","nonce":15705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":15705},{"type":"v1","addr":"172.21.15.114:6807","nonce":15705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":15705},{"type":"v1","addr":"172.21.15.114:6805","nonce":15705}]},"public_addr":"172.21.15.114:6801/15705","cluster_addr":"172.21.15.114:6803/15705","heartbeat_back_addr":"172.21.15.114:6807/15705","heartbeat_front_addr":"172.21.15.114:6805/15705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.014713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.010670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:51.937853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:55.958 INFO:tasks.ceph.ceph_manager.cluster2:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-02-20T07:54:53.662980+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-02-20T07:54:54.212713+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-02-20T07:54:55.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get device_health_metrics pg_num 2021-02-20T07:54:56.261 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2021-02-20T07:54:56.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get rbd pg_num 2021-02-20T07:54:56.579 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 8 2021-02-20T07:54:56.593 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2021-02-20T07:54:56.594 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2021-02-20T07:54:56.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2021-02-20T07:54:56.932 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:56.946 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":18741},{"type":"v1","addr":"172.21.15.106:6827","nonce":18741}]},"active_addr":"172.21.15.106:6827/18741","active_change":"2021-02-20T07:54:49.517258+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3656717332}]},{"addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1655084280}]}]}} 2021-02-20T07:54:56.948 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2021-02-20T07:54:56.948 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2021-02-20T07:54:56.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T07:54:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:57.250 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":11,"fsid":"d75c5c24-a5ff-4862-8e7e-70367438e25f","created":"2021-02-20T07:54:45.550274+0000","modified":"2021-02-20T07:54:56.623484+0000","last_up_change":"2021-02-20T07:54:53.592939+0000","last_in_change":"2021-02-20T07:54:53.592939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:53.662980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:54.212713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6a4ba8fc-9774-4985-a8fd-d8748ef5827d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":15703},{"type":"v1","addr":"172.21.15.114:6817","nonce":15703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":15703},{"type":"v1","addr":"172.21.15.114:6819","nonce":15703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":15703},{"type":"v1","addr":"172.21.15.114:6823","nonce":15703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":15703},{"type":"v1","addr":"172.21.15.114:6821","nonce":15703}]},"public_addr":"172.21.15.114:6817/15703","cluster_addr":"172.21.15.114:6819/15703","heartbeat_back_addr":"172.21.15.114:6823/15703","heartbeat_front_addr":"172.21.15.114:6821/15703","state":["exists","up"]},{"osd":1,"uuid":"2597dc5f-c106-482a-88f0-5cd9bda3b396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":15704},{"type":"v1","addr":"172.21.15.114:6809","nonce":15704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":15704},{"type":"v1","addr":"172.21.15.114:6811","nonce":15704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":15704},{"type":"v1","addr":"172.21.15.114:6815","nonce":15704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":15704},{"type":"v1","addr":"172.21.15.114:6813","nonce":15704}]},"public_addr":"172.21.15.114:6809/15704","cluster_addr":"172.21.15.114:6811/15704","heartbeat_back_addr":"172.21.15.114:6815/15704","heartbeat_front_addr":"172.21.15.114:6813/15704","state":["exists","up"]},{"osd":2,"uuid":"f5b114e5-2b5f-41ad-ba17-4749ce787a24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":15705},{"type":"v1","addr":"172.21.15.114:6801","nonce":15705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":15705},{"type":"v1","addr":"172.21.15.114:6803","nonce":15705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":15705},{"type":"v1","addr":"172.21.15.114:6807","nonce":15705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":15705},{"type":"v1","addr":"172.21.15.114:6805","nonce":15705}]},"public_addr":"172.21.15.114:6801/15705","cluster_addr":"172.21.15.114:6803/15705","heartbeat_back_addr":"172.21.15.114:6807/15705","heartbeat_front_addr":"172.21.15.114:6805/15705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.014713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.010670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:51.937853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:57.264 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2021-02-20T07:54:57.264 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-02-20T07:54:57.563 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:54:57.564 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":11,"fsid":"d75c5c24-a5ff-4862-8e7e-70367438e25f","created":"2021-02-20T07:54:45.550274+0000","modified":"2021-02-20T07:54:56.623484+0000","last_up_change":"2021-02-20T07:54:53.592939+0000","last_in_change":"2021-02-20T07:54:53.592939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-02-20T07:54:53.662980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-02-20T07:54:54.212713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6a4ba8fc-9774-4985-a8fd-d8748ef5827d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":15703},{"type":"v1","addr":"172.21.15.114:6817","nonce":15703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":15703},{"type":"v1","addr":"172.21.15.114:6819","nonce":15703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":15703},{"type":"v1","addr":"172.21.15.114:6823","nonce":15703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":15703},{"type":"v1","addr":"172.21.15.114:6821","nonce":15703}]},"public_addr":"172.21.15.114:6817/15703","cluster_addr":"172.21.15.114:6819/15703","heartbeat_back_addr":"172.21.15.114:6823/15703","heartbeat_front_addr":"172.21.15.114:6821/15703","state":["exists","up"]},{"osd":1,"uuid":"2597dc5f-c106-482a-88f0-5cd9bda3b396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":15704},{"type":"v1","addr":"172.21.15.114:6809","nonce":15704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":15704},{"type":"v1","addr":"172.21.15.114:6811","nonce":15704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":15704},{"type":"v1","addr":"172.21.15.114:6815","nonce":15704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":15704},{"type":"v1","addr":"172.21.15.114:6813","nonce":15704}]},"public_addr":"172.21.15.114:6809/15704","cluster_addr":"172.21.15.114:6811/15704","heartbeat_back_addr":"172.21.15.114:6815/15704","heartbeat_front_addr":"172.21.15.114:6813/15704","state":["exists","up"]},{"osd":2,"uuid":"f5b114e5-2b5f-41ad-ba17-4749ce787a24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":15705},{"type":"v1","addr":"172.21.15.114:6801","nonce":15705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":15705},{"type":"v1","addr":"172.21.15.114:6803","nonce":15705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":15705},{"type":"v1","addr":"172.21.15.114:6807","nonce":15705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":15705},{"type":"v1","addr":"172.21.15.114:6805","nonce":15705}]},"public_addr":"172.21.15.114:6801/15705","cluster_addr":"172.21.15.114:6803/15705","heartbeat_back_addr":"172.21.15.114:6807/15705","heartbeat_front_addr":"172.21.15.114:6805/15705","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.014713+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:52.010670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2021-02-20T07:54:51.937853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-02-20T07:54:57.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2021-02-20T07:54:57.742 INFO:teuthology.orchestra.run.smithi114.stdout:34359738371 2021-02-20T07:54:57.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2021-02-20T07:54:57.903 INFO:teuthology.orchestra.run.smithi114.stdout:34359738370 2021-02-20T07:54:57.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2021-02-20T07:54:58.080 INFO:teuthology.orchestra.run.smithi114.stdout:34359738371 2021-02-20T07:54:58.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T07:54:58.381 INFO:teuthology.orchestra.run.smithi114.stdout:34359738370 2021-02-20T07:54:58.396 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738370 for osd.0 2021-02-20T07:54:59.397 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-02-20T07:54:59.701 INFO:teuthology.orchestra.run.smithi114.stdout:34359738372 2021-02-20T07:54:59.715 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738372 for osd.0 2021-02-20T07:54:59.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2021-02-20T07:55:00.012 INFO:teuthology.orchestra.run.smithi114.stdout:34359738371 2021-02-20T07:55:00.026 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738371 for osd.1 2021-02-20T07:55:00.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2021-02-20T07:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout:34359738371 2021-02-20T07:55:00.342 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738371 for osd.2 2021-02-20T07:55:00.342 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2021-02-20T07:55:00.343 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T07:55:00.640 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:55:00.642 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2021-02-20T07:55:00.654 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T07:54:59.529387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.636260+0000","last_change":"2021-02-20T07:54:55.634896+0000","last_active":"2021-02-20T07:54:55.636260+0000","last_peered":"2021-02-20T07:54:55.636260+0000","last_clean":"2021-02-20T07:54:55.636260+0000","last_became_active":"2021-02-20T07:54:55.634577+0000","last_became_peered":"2021-02-20T07:54:55.634577+0000","last_unstale":"2021-02-20T07:54:55.636260+0000","last_undegraded":"2021-02-20T07:54:55.636260+0000","last_fullsized":"2021-02-20T07:54:55.636260+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.636106+0000","last_change":"2021-02-20T07:54:55.634554+0000","last_active":"2021-02-20T07:54:55.636106+0000","last_peered":"2021-02-20T07:54:55.636106+0000","last_clean":"2021-02-20T07:54:55.636106+0000","last_became_active":"2021-02-20T07:54:55.633726+0000","last_became_peered":"2021-02-20T07:54:55.633726+0000","last_unstale":"2021-02-20T07:54:55.636106+0000","last_undegraded":"2021-02-20T07:54:55.636106+0000","last_fullsized":"2021-02-20T07:54:55.636106+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.635612+0000","last_change":"2021-02-20T07:54:55.633901+0000","last_active":"2021-02-20T07:54:55.635612+0000","last_peered":"2021-02-20T07:54:55.635612+0000","last_clean":"2021-02-20T07:54:55.635612+0000","last_became_active":"2021-02-20T07:54:55.633081+0000","last_became_peered":"2021-02-20T07:54:55.633081+0000","last_unstale":"2021-02-20T07:54:55.635612+0000","last_undegraded":"2021-02-20T07:54:55.635612+0000","last_fullsized":"2021-02-20T07:54:55.635612+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.637544+0000","last_change":"2021-02-20T07:54:55.635804+0000","last_active":"2021-02-20T07:54:55.637544+0000","last_peered":"2021-02-20T07:54:55.637544+0000","last_clean":"2021-02-20T07:54:55.637544+0000","last_became_active":"2021-02-20T07:54:55.635156+0000","last_became_peered":"2021-02-20T07:54:55.635156+0000","last_unstale":"2021-02-20T07:54:55.637544+0000","last_undegraded":"2021-02-20T07:54:55.637544+0000","last_fullsized":"2021-02-20T07:54:55.637544+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.631674+0000","last_change":"2021-02-20T07:54:55.629911+0000","last_active":"2021-02-20T07:54:55.631674+0000","last_peered":"2021-02-20T07:54:55.631674+0000","last_clean":"2021-02-20T07:54:55.631674+0000","last_became_active":"2021-02-20T07:54:55.628580+0000","last_became_peered":"2021-02-20T07:54:55.628580+0000","last_unstale":"2021-02-20T07:54:55.631674+0000","last_undegraded":"2021-02-20T07:54:55.631674+0000","last_fullsized":"2021-02-20T07:54:55.631674+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.635526+0000","last_change":"2021-02-20T07:54:55.633742+0000","last_active":"2021-02-20T07:54:55.635526+0000","last_peered":"2021-02-20T07:54:55.635526+0000","last_clean":"2021-02-20T07:54:55.635526+0000","last_became_active":"2021-02-20T07:54:55.633330+0000","last_became_peered":"2021-02-20T07:54:55.633330+0000","last_unstale":"2021-02-20T07:54:55.635526+0000","last_undegraded":"2021-02-20T07:54:55.635526+0000","last_fullsized":"2021-02-20T07:54:55.635526+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.633664+0000","last_change":"2021-02-20T07:54:55.633032+0000","last_active":"2021-02-20T07:54:55.633664+0000","last_peered":"2021-02-20T07:54:55.633664+0000","last_clean":"2021-02-20T07:54:55.633664+0000","last_became_active":"2021-02-20T07:54:55.632638+0000","last_became_peered":"2021-02-20T07:54:55.632638+0000","last_unstale":"2021-02-20T07:54:55.633664+0000","last_undegraded":"2021-02-20T07:54:55.633664+0000","last_fullsized":"2021-02-20T07:54:55.633664+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.638658+0000","last_change":"2021-02-20T07:54:55.636882+0000","last_active":"2021-02-20T07:54:55.638658+0000","last_peered":"2021-02-20T07:54:55.638658+0000","last_clean":"2021-02-20T07:54:55.638658+0000","last_became_active":"2021-02-20T07:54:55.636360+0000","last_became_peered":"2021-02-20T07:54:55.636360+0000","last_unstale":"2021-02-20T07:54:55.638658+0000","last_undegraded":"2021-02-20T07:54:55.638658+0000","last_fullsized":"2021-02-20T07:54:55.638658+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.630537+0000","last_change":"2021-02-20T07:54:55.629201+0000","last_active":"2021-02-20T07:54:55.630537+0000","last_peered":"2021-02-20T07:54:55.630537+0000","last_clean":"2021-02-20T07:54:55.630537+0000","last_became_active":"2021-02-20T07:54:55.628743+0000","last_became_peered":"2021-02-20T07:54:55.628743+0000","last_unstale":"2021-02-20T07:54:55.630537+0000","last_undegraded":"2021-02-20T07:54:55.630537+0000","last_fullsized":"2021-02-20T07:54:55.630537+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]}]}} 2021-02-20T07:55:00.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-02-20T07:55:00.954 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:55:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2021-02-20T07:55:00.968 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-02-20T07:54:59.529387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.636260+0000","last_change":"2021-02-20T07:54:55.634896+0000","last_active":"2021-02-20T07:54:55.636260+0000","last_peered":"2021-02-20T07:54:55.636260+0000","last_clean":"2021-02-20T07:54:55.636260+0000","last_became_active":"2021-02-20T07:54:55.634577+0000","last_became_peered":"2021-02-20T07:54:55.634577+0000","last_unstale":"2021-02-20T07:54:55.636260+0000","last_undegraded":"2021-02-20T07:54:55.636260+0000","last_fullsized":"2021-02-20T07:54:55.636260+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.636106+0000","last_change":"2021-02-20T07:54:55.634554+0000","last_active":"2021-02-20T07:54:55.636106+0000","last_peered":"2021-02-20T07:54:55.636106+0000","last_clean":"2021-02-20T07:54:55.636106+0000","last_became_active":"2021-02-20T07:54:55.633726+0000","last_became_peered":"2021-02-20T07:54:55.633726+0000","last_unstale":"2021-02-20T07:54:55.636106+0000","last_undegraded":"2021-02-20T07:54:55.636106+0000","last_fullsized":"2021-02-20T07:54:55.636106+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.635612+0000","last_change":"2021-02-20T07:54:55.633901+0000","last_active":"2021-02-20T07:54:55.635612+0000","last_peered":"2021-02-20T07:54:55.635612+0000","last_clean":"2021-02-20T07:54:55.635612+0000","last_became_active":"2021-02-20T07:54:55.633081+0000","last_became_peered":"2021-02-20T07:54:55.633081+0000","last_unstale":"2021-02-20T07:54:55.635612+0000","last_undegraded":"2021-02-20T07:54:55.635612+0000","last_fullsized":"2021-02-20T07:54:55.635612+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.637544+0000","last_change":"2021-02-20T07:54:55.635804+0000","last_active":"2021-02-20T07:54:55.637544+0000","last_peered":"2021-02-20T07:54:55.637544+0000","last_clean":"2021-02-20T07:54:55.637544+0000","last_became_active":"2021-02-20T07:54:55.635156+0000","last_became_peered":"2021-02-20T07:54:55.635156+0000","last_unstale":"2021-02-20T07:54:55.637544+0000","last_undegraded":"2021-02-20T07:54:55.637544+0000","last_fullsized":"2021-02-20T07:54:55.637544+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.631674+0000","last_change":"2021-02-20T07:54:55.629911+0000","last_active":"2021-02-20T07:54:55.631674+0000","last_peered":"2021-02-20T07:54:55.631674+0000","last_clean":"2021-02-20T07:54:55.631674+0000","last_became_active":"2021-02-20T07:54:55.628580+0000","last_became_peered":"2021-02-20T07:54:55.628580+0000","last_unstale":"2021-02-20T07:54:55.631674+0000","last_undegraded":"2021-02-20T07:54:55.631674+0000","last_fullsized":"2021-02-20T07:54:55.631674+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.635526+0000","last_change":"2021-02-20T07:54:55.633742+0000","last_active":"2021-02-20T07:54:55.635526+0000","last_peered":"2021-02-20T07:54:55.635526+0000","last_clean":"2021-02-20T07:54:55.635526+0000","last_became_active":"2021-02-20T07:54:55.633330+0000","last_became_peered":"2021-02-20T07:54:55.633330+0000","last_unstale":"2021-02-20T07:54:55.635526+0000","last_undegraded":"2021-02-20T07:54:55.635526+0000","last_fullsized":"2021-02-20T07:54:55.635526+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.633664+0000","last_change":"2021-02-20T07:54:55.633032+0000","last_active":"2021-02-20T07:54:55.633664+0000","last_peered":"2021-02-20T07:54:55.633664+0000","last_clean":"2021-02-20T07:54:55.633664+0000","last_became_active":"2021-02-20T07:54:55.632638+0000","last_became_peered":"2021-02-20T07:54:55.632638+0000","last_unstale":"2021-02-20T07:54:55.633664+0000","last_undegraded":"2021-02-20T07:54:55.633664+0000","last_fullsized":"2021-02-20T07:54:55.633664+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.638658+0000","last_change":"2021-02-20T07:54:55.636882+0000","last_active":"2021-02-20T07:54:55.638658+0000","last_peered":"2021-02-20T07:54:55.638658+0000","last_clean":"2021-02-20T07:54:55.638658+0000","last_became_active":"2021-02-20T07:54:55.636360+0000","last_became_peered":"2021-02-20T07:54:55.636360+0000","last_unstale":"2021-02-20T07:54:55.638658+0000","last_undegraded":"2021-02-20T07:54:55.638658+0000","last_fullsized":"2021-02-20T07:54:55.638658+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-02-20T07:54:55.630537+0000","last_change":"2021-02-20T07:54:55.629201+0000","last_active":"2021-02-20T07:54:55.630537+0000","last_peered":"2021-02-20T07:54:55.630537+0000","last_clean":"2021-02-20T07:54:55.630537+0000","last_became_active":"2021-02-20T07:54:55.628743+0000","last_became_peered":"2021-02-20T07:54:55.628743+0000","last_unstale":"2021-02-20T07:54:55.630537+0000","last_undegraded":"2021-02-20T07:54:55.630537+0000","last_fullsized":"2021-02-20T07:54:55.630537+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-02-20T07:54:54.600486+0000","last_clean_scrub_stamp":"2021-02-20T07:54:54.600486+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]}]}} 2021-02-20T07:55:00.969 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2021-02-20T07:55:00.970 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2021-02-20T07:55:00.970 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2021-02-20T07:55:00.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2021-02-20T07:55:01.316 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-20T07:55:01.317 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-02-20T07:55:01.333 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2021-02-20T07:55:01.334 INFO:teuthology.run_tasks:Running task exec... 2021-02-20T07:55:01.356 INFO:teuthology.task.exec:Executing custom commands... 2021-02-20T07:55:01.356 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi114.front.sepia.ceph.com 2021-02-20T07:55:01.357 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:01.739 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror 2021-02-20T07:55:01.752 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:02.122 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.0 2021-02-20T07:55:02.136 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:02.517 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.1 2021-02-20T07:55:02.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:02.897 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.2 2021-02-20T07:55:02.910 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:03.284 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.3 2021-02-20T07:55:03.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:03.674 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror 2021-02-20T07:55:03.686 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:04.059 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.0 2021-02-20T07:55:04.072 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:04.441 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.1 2021-02-20T07:55:04.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.2 2021-02-20T07:55:04.839 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-02-20T07:55:05.187 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.mirror.3 2021-02-20T07:55:05.198 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T07:55:05.214 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T07:55:05.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T07:55:05.223 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T07:55:05.236 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T07:55:05.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T07:55:05.238 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T07:55:05.250 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T07:55:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T07:55:05.253 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T07:55:05.266 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T07:55:05.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T07:55:05.269 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T07:55:05.288 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T07:55:05.291 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T07:55:05.291 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T07:55:05.308 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T07:55:05.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T07:55:05.311 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T07:55:05.327 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T07:55:05.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T07:55:05.331 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-02-20T07:55:05.347 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T07:55:05.348 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T07:55:05.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T07:55:05.351 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T07:55:05.371 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613807705 2021-02-20T07:55:05.371 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T07:55:05.372 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-02-20T07:55:05.392 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1613807705 2021-02-20T07:55:05.393 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-02-20T07:55:05.393 INFO:teuthology.run_tasks:Running task workunit... 2021-02-20T07:55:05.423 INFO:tasks.workunit:Pulling workunits from ref d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:55:05.425 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-02-20T07:55:05.425 DEBUG:teuthology.orchestra.run.smithi114:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T07:55:05.426 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2021-02-20T07:55:05.426 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T07:55:05.426 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2021-02-20T07:55:05.427 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T07:55:05.432 INFO:teuthology.orchestra.run.smithi114.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2021-02-20T07:55:05.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:55:05.433 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T07:55:05.482 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-02-20T07:55:05.483 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2021-02-20T07:55:05.529 INFO:tasks.workunit:timeout=6h 2021-02-20T07:55:05.529 INFO:tasks.workunit:cleanup=True 2021-02-20T07:55:05.530 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout d318d42d502ea13c78f2af2499efd19dcc7e174b 2021-02-20T07:55:05.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2021-02-20T07:55:36.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Note: checking out 'd318d42d502ea13c78f2af2499efd19dcc7e174b'. 2021-02-20T07:55:36.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:55:36.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-02-20T07:55:36.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:changes and commit them, and you can discard any commits you make in this 2021-02-20T07:55:36.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:state without impacting any branches by performing another checkout. 2021-02-20T07:55:36.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:55:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:If you want to create a new branch to retain commits you create, you may 2021-02-20T07:55:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:do so (now or later) by using -b with the checkout command again. Example: 2021-02-20T07:55:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:55:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: git checkout -b 2021-02-20T07:55:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:55:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:HEAD is now at d318d42d50 rbd-mirror: delay update snapshot mirror image state by dillaman · Pull Request #39432 · ceph/ceph · GitHub 2021-02-20T07:55:36.809 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2021-02-20T07:55:36.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-02-20T07:55:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T07:55:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-02-20T07:55:37.566 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-02-20T07:55:37.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-02-20T07:55:37.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-02-20T07:55:37.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T07:55:37.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-02-20T07:55:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-02-20T07:55:37.821 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2021-02-20T07:55:37.822 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2021-02-20T07:55:37.872 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2021-02-20T07:55:37.874 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2021-02-20T07:55:37.874 DEBUG:teuthology.orchestra.run.smithi114:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d318d42d502ea13c78f2af2499efd19dcc7e174b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS='' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T07:55:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ IMAGE_COUNT=50 2021-02-20T07:55:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ export LOCKDEP=0 2021-02-20T07:55:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T07:55:37.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2021-02-20T07:55:37.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ type xmlstarlet 2021-02-20T07:55:37.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ XMLSTARLET=xmlstarlet 2021-02-20T07:55:37.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ RBD_MIRROR_INSTANCES=4 2021-02-20T07:55:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CLUSTER1=cluster1 2021-02-20T07:55:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CLUSTER2=cluster2 2021-02-20T07:55:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ PEER_CLUSTER_SUFFIX= 2021-02-20T07:55:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ POOL=mirror 2021-02-20T07:55:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ PARENT_POOL=mirror_parent 2021-02-20T07:55:37.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ NS1=ns1 2021-02-20T07:55:37.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ NS2=ns2 2021-02-20T07:55:37.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ TEMPDIR= 2021-02-20T07:55:37.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ID=mirror 2021-02-20T07:55:37.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2021-02-20T07:55:37.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ MIRROR_USER_ID_PREFIX=mirror. 2021-02-20T07:55:37.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ MIRROR_POOL_MODE=image 2021-02-20T07:55:37.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ MIRROR_IMAGE_MODE=snapshot 2021-02-20T07:55:37.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ export CEPH_ARGS=--id mirror 2021-02-20T07:55:37.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ LAST_MIRROR_INSTANCE=3 2021-02-20T07:55:37.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-02-20T07:55:37.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2021-02-20T07:55:37.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2021-02-20T07:55:37.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_BIN=. 2021-02-20T07:55:37.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_SRC=. 2021-02-20T07:55:37.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -e CMakeCache.txt ] 2021-02-20T07:55:37.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 -gt 0 ] 2021-02-20T07:55:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ setup 2021-02-20T07:55:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local c 2021-02-20T07:55:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ trap cleanup $? INT TERM EXIT 2021-02-20T07:55:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ setup_tempdir 2021-02-20T07:55:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mktemp -d 2021-02-20T07:55:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ TEMPDIR=/tmp/tmp.d0Pc51qzr5 2021-02-20T07:55:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -z 1 ] 2021-02-20T07:55:37.942 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ setup_pools cluster1 cluster2 2021-02-20T07:55:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster2 2021-02-20T07:55:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mon_map_file 2021-02-20T07:55:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mon_addr 2021-02-20T07:55:37.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local admin_key_file 2021-02-20T07:55:37.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid 2021-02-20T07:55:37.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror 64 64 2021-02-20T07:55:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:pool 'mirror' already exists 2021-02-20T07:55:38.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror_parent 64 64 2021-02-20T07:55:39.645 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:pool 'mirror_parent' already exists 2021-02-20T07:55:39.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror 2021-02-20T07:55:42.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror_parent 2021-02-20T07:55:45.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:45.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2021-02-20T07:55:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2021-02-20T07:55:45.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2021-02-20T07:55:45.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2021-02-20T07:55:45.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2021-02-20T07:55:45.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2021-02-20T07:55:45.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -z ] 2021-02-20T07:55:45.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -z ] 2021-02-20T07:55:45.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_add cluster1 mirror cluster2 2021-02-20T07:55:45.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:45.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:45.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local client_cluster=cluster2 2021-02-20T07:55:45.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster2 2021-02-20T07:55:45.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid_var_name 2021-02-20T07:55:45.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:45.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local error_code 2021-02-20T07:55:45.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local peer_uuid 2021-02-20T07:55:45.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:55:45.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2021-02-20T07:55:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_uuid=36d71427-ebf2-4e15-9efd-7b56f41cfe4e 2021-02-20T07:55:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=0 2021-02-20T07:55:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:55:45.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 -eq 17 ] 2021-02-20T07:55:45.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 0 -eq 0 2021-02-20T07:55:45.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:55:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_add cluster1 mirror_parent cluster2 2021-02-20T07:55:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror_parent 2021-02-20T07:55:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local client_cluster=cluster2 2021-02-20T07:55:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster2 2021-02-20T07:55:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid_var_name 2021-02-20T07:55:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local error_code 2021-02-20T07:55:45.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local peer_uuid 2021-02-20T07:55:45.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:55:45.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2021-02-20T07:55:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_uuid=ca6e43d4-992a-45f5-be10-f1be234a2a6b 2021-02-20T07:55:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=0 2021-02-20T07:55:46.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:55:46.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 -eq 17 ] 2021-02-20T07:55:46.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 0 -eq 0 2021-02-20T07:55:46.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:46.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:55:46.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ setup_pools cluster2 cluster1 2021-02-20T07:55:46.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:46.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster1 2021-02-20T07:55:46.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mon_map_file 2021-02-20T07:55:46.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mon_addr 2021-02-20T07:55:46.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local admin_key_file 2021-02-20T07:55:46.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid 2021-02-20T07:55:46.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror 64 64 2021-02-20T07:55:46.672 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:pool 'mirror' already exists 2021-02-20T07:55:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror_parent 64 64 2021-02-20T07:55:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:pool 'mirror_parent' already exists 2021-02-20T07:55:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror 2021-02-20T07:55:50.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror_parent 2021-02-20T07:55:53.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:53.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2021-02-20T07:55:53.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2021-02-20T07:55:53.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2021-02-20T07:55:53.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2021-02-20T07:55:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2021-02-20T07:55:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2021-02-20T07:55:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -z ] 2021-02-20T07:55:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -z ] 2021-02-20T07:55:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_add cluster2 mirror cluster1 2021-02-20T07:55:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:53.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:53.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local client_cluster=cluster1 2021-02-20T07:55:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster1 2021-02-20T07:55:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid_var_name 2021-02-20T07:55:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local error_code 2021-02-20T07:55:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local peer_uuid 2021-02-20T07:55:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:55:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2021-02-20T07:55:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_uuid=74937095-943b-47cc-bd0d-7b50956dfa24 2021-02-20T07:55:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=0 2021-02-20T07:55:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:55:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 -eq 17 ] 2021-02-20T07:55:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 0 -eq 0 2021-02-20T07:55:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:55:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_add cluster2 mirror_parent cluster1 2021-02-20T07:55:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror_parent 2021-02-20T07:55:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local client_cluster=cluster1 2021-02-20T07:55:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local remote_cluster=cluster1 2021-02-20T07:55:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local uuid_var_name 2021-02-20T07:55:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local error_code 2021-02-20T07:55:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local peer_uuid 2021-02-20T07:55:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:55:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2021-02-20T07:55:54.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ peer_uuid=8fbe31e8-7c7a-45bf-b25e-8c2df2ba3a9e 2021-02-20T07:55:54.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=0 2021-02-20T07:55:54.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:55:54.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 -eq 17 ] 2021-02-20T07:55:54.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 0 -eq 0 2021-02-20T07:55:54.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:54.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:55:54.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:55:54.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirrors cluster1 2021-02-20T07:55:54.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:54.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:55:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster1:0 2021-02-20T07:55:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:55:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:55:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:54.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:54.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster1:1 2021-02-20T07:55:54.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:55:54.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:55:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster1:2 2021-02-20T07:55:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:55:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:55:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster1:3 2021-02-20T07:55:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:55:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:55:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:55:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:55:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirrors cluster2 2021-02-20T07:55:54.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:54.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:55:54.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster2:0 2021-02-20T07:55:54.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2:0 2021-02-20T07:55:54.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster2:0 cluster instance 2021-02-20T07:55:54.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster2:0 2021-02-20T07:55:54.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:54.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster2:0 ] 2021-02-20T07:55:54.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster2 2021-02-20T07:55:54.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:54.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:54.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster2:1 2021-02-20T07:55:54.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2:1 2021-02-20T07:55:54.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster2:1 cluster instance 2021-02-20T07:55:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster2:1 2021-02-20T07:55:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster2:1 ] 2021-02-20T07:55:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster2 2021-02-20T07:55:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:54.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:54.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster2:2 2021-02-20T07:55:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2:2 2021-02-20T07:55:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster2:2 cluster instance 2021-02-20T07:55:54.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster2:2 2021-02-20T07:55:54.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.134 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:54.134 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster2:2 ] 2021-02-20T07:55:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster2 2021-02-20T07:55:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:54.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ start_mirror cluster2:3 2021-02-20T07:55:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2:3 2021-02-20T07:55:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster2:3 cluster instance 2021-02-20T07:55:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster2:3 2021-02-20T07:55:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster2:3 ] 2021-02-20T07:55:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster2 2021-02-20T07:55:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster2 2021-02-20T07:55:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:54.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 1 2021-02-20T07:55:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 2021-02-20T07:55:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ testlog TEST: add image and test replay after client crashes 2021-02-20T07:55:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee -a /tmp/tmp.d0Pc51qzr5/rbd-mirror.test.log 2021-02-20T07:55:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ date +%F %T 2021-02-20T07:55:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo 2021-02-20 07:55:54 TEST: add image and test replay after client crashes 2021-02-20T07:55:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:2021-02-20 07:55:54 TEST: add image and test replay after client crashes 2021-02-20T07:55:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=test 2021-02-20T07:55:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror test snapshot 512M 2021-02-20T07:55:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T07:55:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T07:55:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror test 512M 2021-02-20T07:55:54.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T07:55:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 512M ] 2021-02-20T07:55:54.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=512M 2021-02-20T07:55:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/test 2021-02-20T07:55:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T07:55:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror test snapshot 2021-02-20T07:55:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:55:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T07:55:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2021-02-20T07:55:54.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T07:55:54.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T07:55:54.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:54.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:54.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:54.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T07:55:54.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:54.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:54.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:54.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T07:55:54.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:55:54.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:55:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T07:55:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:55.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:55:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:55:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T07:55:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T07:55:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T07:55:55.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T07:55:55.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T07:55:55.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:55.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:55.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local instance=cluster1 2021-02-20T07:55:55.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T07:55:55.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep+ -isleep state 2021-02-20T07:55:55.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 0 2021-02-20T07:55:55.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:55:55.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:55:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:55:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:55:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:55:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:55.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:55.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:55.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:55.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:55.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:55.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:55:55.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:55:55.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:55.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:55:55.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:55.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:55.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:55.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:55.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:55.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:55.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:55.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:55:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:55:55.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:55:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:55:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:55:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:55:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:55:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:55:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:55.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:55:55.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:55.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:55:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:55.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:55:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:55:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:55.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:55:55.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:55.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:55:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:55:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:55:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:55.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:55.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:55.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:55.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:55.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:55:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:55:56.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:56.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:56.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:56.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:55:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:55:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:55:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:55:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:55:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:55:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:55:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:55:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:55:56.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:56.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:56.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:56.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:55:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:56.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:56.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:55:56.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:56.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:56.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:56.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:55:57.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:55:57.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:55:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:55:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:55:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:57.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:55:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:57.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:57.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:57.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:55:57.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:55:57.223 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:57.223 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.223 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:55:57.223 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:57.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:57.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:57.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:55:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:57.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:55:57.344 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:57.344 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:55:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:55:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:55:57.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:57.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:57.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:55:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:55:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:55:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:57.353 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:57.353 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:55:57.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:57.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:57.375 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:57.375 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:55:57.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:57.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:57.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:57.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:55:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:55:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:55:57.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:57.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:57.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:57.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:57.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:57.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:57.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:57.474 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:55:57.474 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:55:57.474 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:57.474 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:55:57.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:57.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:57.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:57.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:57.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:57.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:57.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:55:57.491 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:55:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:57.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:57.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:55:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:55:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:55:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:55:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:55:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:57.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:57.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:55:57.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:55:57.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:57.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:57.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:55:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:55:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:57.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:57.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:57.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:57.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:55:57.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:57.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:57.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:55:57.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:55:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:55:57.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:55:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:55:57.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:57.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:57.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:57.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:55:59.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:55:59.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:55:59.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:55:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:55:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:55:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:55:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:55:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:55:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:55:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:55:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:55:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:55:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:55:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:55:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:55:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:59.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:59.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:59.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:55:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:55:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:55:59.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:59.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:59.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:59.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:59.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:59.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:59.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:55:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:55:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:55:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:55:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:55:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:55:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:55:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:55:59.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:59.850 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:59.850 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:55:59.850 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:55:59.948 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:55:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:55:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:55:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:55:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:55:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:55:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:55:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:55:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:55:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:59.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:59.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:55:59.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:55:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:55:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:55:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:55:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:55:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:55:59.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:55:59.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:55:59.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:59.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:55:59.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:55:59.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:55:59.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:55:59.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:55:59.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:55:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:55:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:56:00.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:00.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:00.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:00.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:56:00.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:56:00.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:00.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:00.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:56:00.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:00.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:00.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:00.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:00.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:00.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:00.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:00.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:00.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:00.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:00.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:56:00.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:56:00.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:00.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:00.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:56:00.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:00.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:00.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:00.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:00.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:00.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:56:00.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:56:00.178 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 4 2021-02-20T07:56:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:56:04.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:56:04.183 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:56:04.183 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:56:04.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:04.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:56:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:04.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:04.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:56:04.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:56:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:56:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:04.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:04.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:56:04.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:04.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:04.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:56:04.303 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:04.303 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:04.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:56:04.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:56:04.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:04.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:56:04.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:04.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:04.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:56:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:56:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:56:04.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:04.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:04.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.315 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:04.315 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:04.315 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:56:04.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:04.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:04.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:04.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:56:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:04.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:04.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:04.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:56:04.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:56:04.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:04.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:56:04.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:04.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:04.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:04.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:04.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:04.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:04.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:04.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:56:04.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:56:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:56:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:04.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:04.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:04.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:04.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:04.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:56:04.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:04.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:04.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:04.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:56:04.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:04.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:04.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:04.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:56:04.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:56:04.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:04.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:56:04.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:04.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:04.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:04.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:56:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:56:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:56:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:04.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:04.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:04.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:04.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:04.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:56:04.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:04.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:56:04.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:04.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:04.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:04.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T07:56:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:56:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:56:12.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:56:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:56:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:56:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:12.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:12.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:12.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:12.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:12.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:56:12.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:56:12.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:12.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:12.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:56:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:12.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:12.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:56:12.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:12.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:12.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:12.681 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:56:12.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:12.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:12.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:12.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:56:12.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:56:12.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:12.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:12.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:56:12.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:12.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:12.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:12.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:12.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:12.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:56:12.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:56:12.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:12.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:12.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:56:12.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:12.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:12.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:56:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:56:12.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:56:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T07:56:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T07:56:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T07:56:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T07:56:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T07:56:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T07:56:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:56:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name= 2021-02-20T07:56:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 1 10 2021-02-20T07:56:12.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T07:56:12.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:56:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T07:56:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=6 2021-02-20T07:56:12.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:56:12.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T07:56:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 25560064 bytes 2021-02-20T07:56:13.562 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 56640512 bytes 2021-02-20T07:56:13.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 116628480 bytes 2021-02-20T07:56:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 156125184 bytes 2021-02-20T07:56:14.229 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 168256512 bytes 2021-02-20T07:56:14.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 219179008 bytes 2021-02-20T07:56:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 257765376 bytes 2021-02-20T07:56:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 261835776 bytes 2021-02-20T07:56:15.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 303291392 bytes 2021-02-20T07:56:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 361138176 bytes 2021-02-20T07:56:15.431 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T07:56:15.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T07:56:15.433 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T07:56:15.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T07:56:15.430+0000 7f225d115700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 19016) UID: 1000 2021-02-20T07:56:15.645 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 410077184 bytes 2021-02-20T07:56:15.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T07:56:15.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 449753088 bytes 2021-02-20T07:56:16.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 503583744 bytes 2021-02-20T07:56:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 548795392 bytes 2021-02-20T07:56:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 575903744 bytes 2021-02-20T07:56:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 583093248 bytes 2021-02-20T07:56:17.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 656680960 bytes 2021-02-20T07:56:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 661725184 bytes 2021-02-20T07:56:17.433 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T07:56:17.540 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T07:56:17.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:56:17.542 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T07:56:17.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T07:56:17.543 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T07:56:17.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 717974528 bytes 2021-02-20T07:56:17.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T07:56:17.548 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T07:56:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 755166208 bytes 2021-02-20T07:56:18.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 822364160 bytes 2021-02-20T07:56:18.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 859292672 bytes 2021-02-20T07:56:18.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 859915264 bytes 2021-02-20T07:56:18.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 885520384 bytes 2021-02-20T07:56:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 911262720 bytes 2021-02-20T07:56:18.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T07:56:18.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:56:18.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T07:56:18.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:56:18.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:56:18.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:18.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:18.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:18.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:56:18.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:56:18.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:56:18.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:56:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:56:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:56:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:56:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:56:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T07:56:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T07:56:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:56:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T07:56:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:20.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T07:56:20.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:56:20.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T07:56:20.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613807754,"replay_state":"idle"} 2021-02-20T07:56:20.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.1 on smithi114 2021-02-20T07:56:20.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:56:07 2021-02-20T07:56:20.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T07:56:20.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T07:56:20.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: down+unknown 2021-02-20T07:56:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: status not found 2021-02-20T07:56:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20T07:56:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:20.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T07:56:20.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:20.015 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1613807754,"replay_state":"idle"} 2021-02-20T07:56:20.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:56:20.017 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:56:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T07:56:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:56:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.1 on smithi114 2021-02-20T07:56:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:20.020 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:56:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 07:56:07 2021-02-20T07:56:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 07:56:07 2021-02-20T07:56:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:56:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 07:56:07 2021-02-20T07:56:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T07:56:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T07:56:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:56:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T07:56:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T07:56:20.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T07:56:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T07:56:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T07:56:20.208 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 07:56:07 2021-02-20T07:56:20.208 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T07:56:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T07:56:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:56:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T07:56:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:56:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:56:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:56:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap1 2021-02-20T07:56:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap1 2021-02-20T07:56:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:56:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1 2021-02-20T07:56:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2021-02-20T07:56:50.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T07:56:50.692 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:50.692 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:50.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:50.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T07:56:50.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:50.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:50.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:50.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T07:56:50.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:56:50.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:56:51.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40075678912871626) > weight (0.25) 2021-02-20T07:56:51.428 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T07:56:51.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T07:56:51.429 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T07:56:51.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T07:56:51.430+0000 7f5aa357c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 18994) UID: 1000 2021-02-20T07:56:51.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T07:56:51.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T07:56:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:56:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:56:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T07:56:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T07:56:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T07:56:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T07:56:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T07:56:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local instance=cluster1 2021-02-20T07:56:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T07:56:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T07:56:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T07:56:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:56:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:56:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:56:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:56:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:56:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:51.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:51.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:51.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:56:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:56:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:56:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:56:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:56:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:56:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:56:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:56:51.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:51.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:51.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:56:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:56:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:56:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:51.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:51.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:51.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:51.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:51.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:56:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:51.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:51.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:51.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:56:51.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2021-02-20T07:56:51.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:56:51.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:56:51.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:51.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:56:51.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:51.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:51.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:51.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:51.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:56:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:56:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:56:51.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:51.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:51.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:51.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:51.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:51.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:56:51.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:51.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:51.951 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:51.951 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:56:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:52.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:56:52.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:56:52.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:52.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:52.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:56:52.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:52.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:52.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:52.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:52.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:56:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:56:52.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:52.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:52.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:56:52.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:52.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:52.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:52.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:52.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:52.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:52.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:52.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:56:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:56:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:56:53.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:56:53.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:56:53.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:56:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:56:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:56:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:53.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:53.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:56:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:56:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:56:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:56:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:56:53.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:53.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:53.302 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:53.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:56:53.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:56:53.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:53.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:56:53.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:53.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:53.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:53.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:56:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:56:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:56:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:53.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:53.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:53.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:53.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:53.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:56:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:56:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2021-02-20T07:56:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:56:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:56:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:56:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:53.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:53.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:53.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:53.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:56:53.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:56:53.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:53.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:56:53.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:53.426 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:53.426 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:56:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:56:53.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:53.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:53.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:56:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:56:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:56:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:53.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:53.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:53.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:56:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:56:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:56:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:56:53.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:56:53.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:53.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:56:55.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:56:55.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:56:55.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:56:55.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:56:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:56:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:55.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:55.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:55.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:56:55.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:56:55.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:55.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:56:55.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:56:55.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:55.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:55.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:56:55.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:55.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:56:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:56:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:56:55.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:55.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:55.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:56:55.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:56:55.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:55.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:55.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:56:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:56:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:56:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:56:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:56:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:56:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:56:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:56:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:55.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:55.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:56:55.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:56:55.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:56:55.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:56:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:56:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2021-02-20T07:56:55.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:56:55.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:56:55.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:55.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:56:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:56:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:56:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:56:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:56:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:56:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:55.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:55.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:56:55.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:56:55.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:56:55.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:55.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:55.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:56:55.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:56:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:56.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:56:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:56:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:56:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:56:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:56:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:56.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:56:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:56.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:56:56.004 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:56:56.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:56:56.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:56:56.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:56:56.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:56:56.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:56:56.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:56:56.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:56:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:56:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:56:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:56:56.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:56:56.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:56:56.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:56:56.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:56:56.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:56:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:56:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:56:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:56:56.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 4 2021-02-20T07:56:56.431 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T07:56:56.431 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T07:56:56.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:56:56.433 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T07:56:56.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T07:56:56.434 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T07:56:56.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T07:56:56.438 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T07:57:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:57:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:00.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:00.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:00.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:00.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:57:00.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:00.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:57:00.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:00.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:00.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:00.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:57:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:00.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:57:00.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:57:00.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:00.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:57:00.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:00.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:00.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:00.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:57:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:57:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:57:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:00.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:57:00.261 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:57:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:57:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:57:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:57:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:57:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:57:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:57:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:00.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:00.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:00.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:00.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:00.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:00.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:00.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:00.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:57:00.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:00.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:00.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:00.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:57:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:57:00.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:57:00.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:00.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:57:00.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:00.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:00.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:57:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:57:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:57:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:00.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:00.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:00.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:00.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:00.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:57:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:57:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T07:57:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:57:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:57:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:57:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:57:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:57:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:08.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:08.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:57:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:57:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:57:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:08.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:08.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:08.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:08.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:57:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:57:08.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:08.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:08.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:57:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:57:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:57:08.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:08.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:08.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:08.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:08.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:08.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:08.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:08.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:57:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:57:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:57:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:08.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:57:08.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:08.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:08.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:08.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:57:08.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:08.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:08.835 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:57:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:57:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:57:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:57:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:57:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:57:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:08.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:08.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:08.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:08.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:08.844 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:08.844 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:57:08.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:08.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:08.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:08.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:57:08.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:08.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:08.957 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:57:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:57:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:08.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:57:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:08.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:08.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:08.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:57:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:57:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:08.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:08.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:57:08.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:08.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:57:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:08.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:57:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:09.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T07:57:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:57:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:57:17.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:57:17.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:57:17.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:17.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:57:17.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:17.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:17.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:17.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:17.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:17.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:17.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:17.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:57:17.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:57:17.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:17.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:57:17.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:17.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:17.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:17.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:17.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:17.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:57:17.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:57:17.211 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:17.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:17.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T07:57:17.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T07:57:17.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:17.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T07:57:17.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T07:57:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T07:57:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T07:57:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T07:57:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:17.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T07:57:17.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:17.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:17.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T07:57:17.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T07:57:17.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T07:57:17.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:17.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:17.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T07:57:17.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T07:57:17.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:17.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:17.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T07:57:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T07:57:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T07:57:17.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:17.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:17.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:17.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:17.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:17.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T07:57:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T07:57:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T07:57:17.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T07:57:17.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:17.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:17.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T07:57:17.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:17.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:17.341 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T07:57:17.341 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T07:57:17.341 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T07:57:17.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:17.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:17.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T07:57:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T07:57:17.447 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:17.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:17.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:17.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T07:57:17.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T07:57:17.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:17.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T07:57:17.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:17.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:17.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.452 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.452 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:17.452 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:17.452 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T07:57:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T07:57:17.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:17.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:17.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T07:57:17.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T07:57:17.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:17.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:17.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:17.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:17.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T07:57:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T07:57:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T07:57:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T07:57:17.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:17.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:17.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T07:57:17.473 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T07:57:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T07:57:17.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T07:57:17.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T07:57:17.572 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T07:57:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:57:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:57:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:57:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:57:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:57:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:57:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:25.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:57:25.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:25.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:25.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:57:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:25.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:57:25.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:57:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:57:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:57:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:57:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:57:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:57:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:57:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:57:25.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:57:25.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:57:25.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:57:25.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:57:25.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:57:25.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:25.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:25.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:57:25.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:57:25.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:25.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T07:57:25.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T07:57:25.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T07:57:25.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T07:57:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T07:57:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T07:57:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T07:57:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:57:25.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:25.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:25.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:25.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T07:57:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T07:57:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T07:57:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:57:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T07:57:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T07:57:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:57:25.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:57:25.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T07:57:25.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T07:57:25.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:57:25.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:25.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T07:57:25.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:25.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:25.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:25.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T07:57:26.600 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 6 2021-02-20T07:57:26.611 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:57:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:57:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T07:57:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:26.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T07:57:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=6 2021-02-20T07:57:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:57:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T07:57:26.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:26.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:26.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:26.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:26.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:26.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:57:26.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:57:26.946 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=4 2021-02-20T07:57:26.946 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 6 = 4 2021-02-20T07:57:26.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T07:57:27.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:27.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:27.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:27.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:27.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:27.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:57:27.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:27.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:57:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=4 2021-02-20T07:57:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 6 = 4 2021-02-20T07:57:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T07:57:28.220 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:28.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:28.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:28.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:28.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:28.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:57:28.222 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:28.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:57:28.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:57:28.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:57:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:57:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:57:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:57:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 6 = 2021-02-20T07:57:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T07:57:29.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:29.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:29.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:29.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:29.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:29.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:57:29.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:57:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:57:29.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:57:29.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:57:29.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:57:29.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:57:29.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 6 = 2021-02-20T07:57:29.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:57:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:57:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:57:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:57:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:57:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=6 2021-02-20T07:57:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 6 = 6 2021-02-20T07:57:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2021-02-20T07:57:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1 2021-02-20T07:57:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:57:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:57:33.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2021-02-20T07:57:33.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T07:57:33.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T07:57:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T07:57:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 2 2021-02-20T07:57:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T07:57:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T07:57:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T07:57:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T07:57:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T07:57:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T07:57:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:57:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:57:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:57:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T07:57:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T07:57:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T07:57:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:57:33.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T07:57:33.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:33.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:57:33.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T07:57:33.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:33.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:33.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1 2021-02-20T07:57:33.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T07:57:33.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:57:33.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:57:33.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:57:33.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:57:33.529 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:57:33.530+0000 7ff4e7c9e700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2021-02-20T07:57:33.530 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:57:33.530+0000 7ff4e9ca2700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2021-02-20T07:57:33.530 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T07:57:33.530+0000 7ff4eb4a5700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2021-02-20T07:57:33.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:57:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:57:34.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:57:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:57:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:57:35.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap1-clean 2021-02-20T07:57:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap1-clean 2021-02-20T07:57:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:35.198 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1-clean 2021-02-20T07:57:35.198 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T07:57:35.682 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T07:57:35.683 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:35.683 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:35.683 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:35.683 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T07:57:35.684 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T07:57:35.684 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:57:35.685 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T07:57:36.564 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 42536960 bytes 2021-02-20T07:57:36.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 66568192 bytes 2021-02-20T07:57:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 114024448 bytes 2021-02-20T07:57:37.682 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 144114688 bytes 2021-02-20T07:57:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 144714752 bytes 2021-02-20T07:57:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 157495296 bytes 2021-02-20T07:57:38.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 200288256 bytes 2021-02-20T07:57:38.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 236235776 bytes 2021-02-20T07:57:38.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 298664960 bytes 2021-02-20T07:57:39.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 373166080 bytes 2021-02-20T07:57:39.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 409516032 bytes 2021-02-20T07:57:39.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 449106944 bytes 2021-02-20T07:57:40.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 473340928 bytes 2021-02-20T07:57:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 484601856 bytes 2021-02-20T07:57:40.489 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 523060224 bytes 2021-02-20T07:57:40.688 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T07:57:40.689 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:57:40.689 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T07:57:40.689 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:40.690 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:57:40.690 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:40.690 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:40.690 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:40.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:57:40.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:57:40.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:57:40.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:57:41.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:57:41.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:41.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:41.692 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:41.692 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:57:41.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:57:41.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:57:41.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.694 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:57:41.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T07:57:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T07:57:41.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:57:41.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T07:57:41.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":45794611.2,"bytes_per_snapshot":457946112.0,"local_snapshot_timestamp":1613807846,"remote_snapshot_timestamp":1613807846,"replay_state":"idle"} 2021-02-20T07:57:41.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T07:57:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:57:35 2021-02-20T07:57:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T07:57:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T07:57:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T07:57:41.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T07:57:41.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:57:35 2021-02-20T07:57:41.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T07:57:41.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":45794611.2,"bytes_per_snapshot":457946112.0,"local_snapshot_timestamp":1613807846,"remote_snapshot_timestamp":1613807846,"replay_state":"idle"} 2021-02-20T07:57:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:57:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + echo up+replayinggrep 2021-02-20T07:57:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: ^up+ 2021-02-20T07:57:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:57:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T07:57:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:57:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 07:57:35 2021-02-20T07:57:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 07:57:35 2021-02-20T07:57:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 07:57:35 2021-02-20T07:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T07:57:41.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T07:57:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:57:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T07:57:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T07:57:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T07:57:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T07:57:42.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T07:57:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 07:57:35 2021-02-20T07:57:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T07:57:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T07:57:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:57:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T07:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap2 2021-02-20T07:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap2 2021-02-20T07:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:57:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:57:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:57:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2 2021-02-20T07:57:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2021-02-20T07:57:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:58:00.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9492755652521354) > weight (0.25) 2021-02-20T07:58:00.441 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T07:58:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T07:58:00.442 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T07:58:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T07:58:00.442+0000 7f6f0325a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 21629) UID: 1000 2021-02-20T07:58:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T07:58:01.443 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T07:58:01.443 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T07:58:01.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:58:01.444 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T07:58:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T07:58:01.445 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T07:58:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T07:58:01.448 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T07:58:04.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4612004735309797) > weight (0.25) 2021-02-20T07:58:04.550 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T07:58:04.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T07:58:04.551 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T07:58:04.553 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T07:58:04.554+0000 7fb66c41b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 19052) UID: 1000 2021-02-20T07:58:04.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T07:58:08.553 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T07:58:08.553 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T07:58:08.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:58:08.554 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T07:58:08.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T07:58:08.555 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T07:58:08.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T07:58:08.558 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T07:58:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T07:58:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T07:58:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:11.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T07:58:11.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:58:11.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:58:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T07:58:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T07:58:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T07:58:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T07:58:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T07:58:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T07:58:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift+ 2021-02-20T07:58:12.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:12.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T07:58:12.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T07:58:12.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T07:58:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T07:58:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:58:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:58:12.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:58:12.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:58:12.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:58:12.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:58:12.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:58:12.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:58:12.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:58:12.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:58:12.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:58:12.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:58:12.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:58:12.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:58:12.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:58:12.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:58:12.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:58:12.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:58:12.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:58:12.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:58:12.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:58:12.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:58:12.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:58:12.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:58:12.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:58:12.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:58:12.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:58:12.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:58:12.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:58:12.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:58:12.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:58:12.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:58:12.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:58:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T07:58:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T07:58:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T07:58:12.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T07:58:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T07:58:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T07:58:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:58:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T07:58:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T07:58:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T07:58:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:58:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T07:58:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T07:58:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:58:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:58:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T07:58:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T07:58:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:58:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T07:58:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T07:58:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T07:58:13.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 10 2021-02-20T07:58:13.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:58:13.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:13.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:13.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:58:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T07:58:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T07:58:13.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=10 2021-02-20T07:58:13.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:58:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T07:58:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:13.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:13.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=6 2021-02-20T07:58:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 6 2021-02-20T07:58:14.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T07:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:14.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:14.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:14.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:14.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:14.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:14.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=6 2021-02-20T07:58:14.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 6 2021-02-20T07:58:14.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T07:58:15.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:15.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:58:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:58:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:58:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:58:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:58:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 2021-02-20T07:58:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T07:58:17.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:17.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:17.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:17.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:58:17.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:58:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:58:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:58:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:58:17.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 2021-02-20T07:58:17.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:58:19.140 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:58:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:58:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:58:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:58:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:58:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 2021-02-20T07:58:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:58:21.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:21.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:21.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:21.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:21.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:58:21.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:58:21.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:58:21.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:58:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=10 2021-02-20T07:58:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 10 = 10 2021-02-20T07:58:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2021-02-20T07:58:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:21.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:21.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2 2021-02-20T07:58:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:58:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:58:22.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2021-02-20T07:58:22.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T07:58:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T07:58:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T07:58:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 4 2021-02-20T07:58:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T07:58:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T07:58:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T07:58:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T07:58:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T07:58:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T07:58:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:58:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 07:57:33 2021 2021-02-20T07:58:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:58:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T07:58:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T07:58:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T07:58:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:58:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T07:58:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap1-clean ] 2021-02-20T07:58:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T07:58:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1-clean 2021-02-20T07:58:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T07:58:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:58:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:22.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:58:23.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:58:23.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:58:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:24.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:24.498 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T07:58:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2 2021-02-20T07:58:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T07:58:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:58:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:24.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:58:25.294 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:58:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:26.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:58:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:26.584 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:58:26.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:26.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap2-clean 2021-02-20T07:58:26.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap2-clean 2021-02-20T07:58:26.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:26.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:26.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:26.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2-clean 2021-02-20T07:58:26.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T07:58:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T07:58:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T07:58:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T07:58:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:58:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T07:58:28.728 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 68527104 bytes 2021-02-20T07:58:29.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 80988160 bytes 2021-02-20T07:58:29.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 100274176 bytes 2021-02-20T07:58:29.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 157881344 bytes 2021-02-20T07:58:30.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 278548480 bytes 2021-02-20T07:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 327017472 bytes 2021-02-20T07:58:31.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 345572352 bytes 2021-02-20T07:58:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 404481024 bytes 2021-02-20T07:58:31.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 447581184 bytes 2021-02-20T07:58:32.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 511443968 bytes 2021-02-20T07:58:32.559 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 591893504 bytes 2021-02-20T07:58:32.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 618907648 bytes 2021-02-20T07:58:33.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 648517632 bytes 2021-02-20T07:58:33.516 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 702543872 bytes 2021-02-20T07:58:33.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 759663616 bytes 2021-02-20T07:58:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 781291520 bytes 2021-02-20T07:58:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 871826432 bytes 2021-02-20T07:58:35.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 942130176 bytes 2021-02-20T07:58:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 947898368 bytes 2021-02-20T07:58:35.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 968299520 bytes 2021-02-20T07:58:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 1073087488 bytes 2021-02-20T07:58:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1121577984 bytes 2021-02-20T07:58:36.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T07:58:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:58:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T07:58:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:58:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:58:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:58:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:58:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:58:37.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:58:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:58:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:58:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:58:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:58:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test+ 2021-02-20T07:58:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T07:58:37.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:58:37.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T07:58:37.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":12027938.13,"bytes_per_snapshot":409392128.0,"local_snapshot_timestamp":1613807893,"remote_snapshot_timestamp":1613807893,"replay_state":"idle"} 2021-02-20T07:58:37.696 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T07:58:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:58:35 2021-02-20T07:58:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T07:58:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T07:58:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T07:58:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T07:58:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:58:35 2021-02-20T07:58:37.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T07:58:37.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":12027938.13,"bytes_per_snapshot":409392128.0,"local_snapshot_timestamp":1613807893,"remote_snapshot_timestamp":1613807893,"replay_state":"idle"} 2021-02-20T07:58:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:58:37.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:58:37.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T07:58:37.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:58:37.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T07:58:37.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:58:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 07:58:35 2021-02-20T07:58:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 07:58:35 2021-02-20T07:58:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:58:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:37.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:37.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:58:37.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 07:58:35 2021-02-20T07:58:37.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T07:58:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T07:58:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:58:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T07:58:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T07:58:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T07:58:37.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T07:58:37.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T07:58:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 07:58:35 2021-02-20T07:58:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T07:58:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T07:58:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:58:37.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T07:58:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:58:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:37.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:58:37.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap3 2021-02-20T07:58:37.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap3 2021-02-20T07:58:37.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:58:37.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:58:37.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:58:37.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3 2021-02-20T07:58:37.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2021-02-20T07:59:06.718 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T07:59:06.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:06.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T07:59:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:06.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T07:59:06.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:59:06.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:59:07.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T07:59:07.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:07.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:07.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:07.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T07:59:07.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T07:59:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T07:59:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T07:59:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T07:59:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:59:07.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:07.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local instance=cluster1 2021-02-20T07:59:07.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T07:59:07.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T07:59:07.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T07:59:07.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T07:59:07.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T07:59:07.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T07:59:07.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T07:59:07.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T07:59:07.725 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:59:07.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T07:59:07.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:59:07.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T07:59:07.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:59:07.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:59:07.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:59:07.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:59:07.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T07:59:07.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:59:07.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:59:07.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:59:07.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T07:59:07.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T07:59:07.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T07:59:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T07:59:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T07:59:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T07:59:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T07:59:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T07:59:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:59:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T07:59:07.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T07:59:07.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T07:59:07.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T07:59:07.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T07:59:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T07:59:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:59:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:59:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T07:59:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T07:59:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T07:59:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + echo "state": "Replaying" 2021-02-20T07:59:07.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i Replaying 2021-02-20T07:59:07.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T07:59:07.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T07:59:07.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T07:59:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:59:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T07:59:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T07:59:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T07:59:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T07:59:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T07:59:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T07:59:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T07:59:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:59:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:59:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T07:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T07:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T07:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T07:59:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 14 2021-02-20T07:59:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:59:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T07:59:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T07:59:08.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:08.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T07:59:08.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=14 2021-02-20T07:59:08.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:59:08.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T07:59:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:09.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:09.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:09.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:09.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:59:09.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:59:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:59:09.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:59:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:59:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:59:09.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:59:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 14 = 2021-02-20T07:59:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T07:59:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:09.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:09.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:09.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:59:09.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:09.570 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:59:09.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:59:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:59:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:59:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:59:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:59:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 14 = 2021-02-20T07:59:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T07:59:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:59:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:59:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:59:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:59:10.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:59:10.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:59:10.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:59:10.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 14 = 2021-02-20T07:59:10.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T07:59:12.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:12.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:59:12.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:12.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T07:59:12.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:59:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T07:59:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T07:59:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T07:59:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T07:59:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 14 = 2021-02-20T07:59:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T07:59:13.750 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:13.750+0000 7f508e15e700 -1 reset not still connected to 0x55ac72e79930 2021-02-20T07:59:13.751 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:13.750+0000 7f508e15e700 -1 reset not still connected to 0x55ac72e7a490 2021-02-20T07:59:13.751 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:13.750+0000 7f508e15e700 -1 reset not still connected to 0x55ac72e7a560 2021-02-20T07:59:13.751 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:13.750+0000 7f508e15e700 -1 reset not still connected to 0x55ac72e7aa40 2021-02-20T07:59:13.751 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:13.750+0000 7f508e15e700 -1 reset not still connected to 0x55ac72e7b8e0 2021-02-20T07:59:14.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:14.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:14.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:14.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:14.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T07:59:14.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T07:59:14.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T07:59:14.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T07:59:14.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=14 2021-02-20T07:59:14.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 14 = 14 2021-02-20T07:59:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2021-02-20T07:59:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:14.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3 2021-02-20T07:59:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T07:59:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:59:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2021-02-20T07:59:15.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T07:59:15.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T07:59:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T07:59:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 7 2021-02-20T07:59:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T07:59:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T07:59:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T07:59:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, operations, non-primary 2021-02-20T07:59:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: snap-trash 2021-02-20T07:59:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T07:59:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:59:15.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 07:57:33 2021 2021-02-20T07:59:15.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T07:59:15.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T07:59:15.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T07:59:15.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T07:59:15.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:59:15.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T07:59:15.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:15.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap2-clean ] 2021-02-20T07:59:15.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T07:59:15.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2-clean 2021-02-20T07:59:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T07:59:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:59:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:59:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:59:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:16.959 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T07:59:16.962+0000 7f5083148700 -1 bdev(0x55ac6a644000 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 3 2021-02-20T07:59:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:59:17.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:17.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:17.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:17.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T07:59:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3 2021-02-20T07:59:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T07:59:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:59:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T07:59:18.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:59:18.512 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:19.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T07:59:19.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:19.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T07:59:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap3-clean 2021-02-20T07:59:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap3-clean 2021-02-20T07:59:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3-clean 2021-02-20T07:59:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T07:59:20.559 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T07:59:20.560 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T07:59:20.560 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T07:59:20.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T07:59:20.562+0000 7f968c357700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 21299) UID: 1000 2021-02-20T07:59:20.761 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T07:59:20.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T07:59:20.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:20.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:20.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:20.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T07:59:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=13 2021-02-20T07:59:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T07:59:20.775 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T07:59:21.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 28783616 bytes 2021-02-20T07:59:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 79868928 bytes 2021-02-20T07:59:22.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 105744384 bytes 2021-02-20T07:59:23.280 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 160694272 bytes 2021-02-20T07:59:23.434 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 161292288 bytes 2021-02-20T07:59:23.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 224029696 bytes 2021-02-20T07:59:24.259 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 282977280 bytes 2021-02-20T07:59:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 312587264 bytes 2021-02-20T07:59:24.561 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T07:59:24.561 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T07:59:24.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T07:59:24.562 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T07:59:24.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T07:59:24.563 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T07:59:24.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T07:59:24.566 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T07:59:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 376004608 bytes 2021-02-20T07:59:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 380456960 bytes 2021-02-20T07:59:25.439 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 444372992 bytes 2021-02-20T07:59:26.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 529602560 bytes 2021-02-20T07:59:26.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 645507072 bytes 2021-02-20T07:59:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 730135552 bytes 2021-02-20T07:59:27.532 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 763441152 bytes 2021-02-20T07:59:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 794375168 bytes 2021-02-20T07:59:27.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 794986496 bytes 2021-02-20T07:59:27.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 798023680 bytes 2021-02-20T07:59:28.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 841714688 bytes 2021-02-20T07:59:28.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 951587840 bytes 2021-02-20T07:59:29.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 977137664 bytes 2021-02-20T07:59:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1034918912 bytes 2021-02-20T07:59:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 1056973824 bytes 2021-02-20T07:59:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1095200768 bytes 2021-02-20T07:59:30.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1130086400 bytes 2021-02-20T07:59:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 1162813440 bytes 2021-02-20T07:59:30.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2700 IOs, 1209561088 bytes 2021-02-20T07:59:31.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2800 IOs, 1233805312 bytes 2021-02-20T07:59:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2900 IOs, 1294716928 bytes 2021-02-20T07:59:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3000 IOs, 1327242240 bytes 2021-02-20T07:59:32.401 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3100 IOs, 1390713856 bytes 2021-02-20T07:59:32.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3200 IOs, 1408704512 bytes 2021-02-20T07:59:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3300 IOs, 1409778688 bytes 2021-02-20T07:59:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3400 IOs, 1432203264 bytes 2021-02-20T07:59:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3500 IOs, 1440681984 bytes 2021-02-20T07:59:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3600 IOs, 1460354048 bytes 2021-02-20T07:59:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3700 IOs, 1473213440 bytes 2021-02-20T07:59:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3800 IOs, 1494332416 bytes 2021-02-20T07:59:33.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T07:59:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T07:59:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T07:59:33.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:33.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:59:33.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:59:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:59:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:59:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T07:59:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T07:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:59:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T07:59:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T07:59:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:59:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T07:59:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T07:59:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T07:59:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T07:59:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":409392128.0,"local_snapshot_timestamp":1613807893,"remote_snapshot_timestamp":1613807893,"replay_state":"idle"} 2021-02-20T07:59:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T07:59:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:59:05 2021-02-20T07:59:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T07:59:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T07:59:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T07:59:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T07:59:34.855 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 07:59:05 2021-02-20T07:59:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T07:59:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":409392128.0,"local_snapshot_timestamp":1613807893,"remote_snapshot_timestamp":1613807893,"replay_state":"idle"} 2021-02-20T07:59:34.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T07:59:34.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:59:34.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T07:59:34.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:59:34.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T07:59:34.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T07:59:34.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 07:59:05 2021-02-20T07:59:34.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 07:59:05 2021-02-20T07:59:34.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T07:59:34.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:34.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T07:59:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 07:59:05 2021-02-20T07:59:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T07:59:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T07:59:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T07:59:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T07:59:34.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T07:59:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T07:59:34.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T07:59:34.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T07:59:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 07:59:05 2021-02-20T07:59:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T07:59:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T07:59:34.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T07:59:34.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T07:59:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T07:59:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T07:59:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap4 2021-02-20T07:59:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap4 2021-02-20T07:59:34.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T07:59:34.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T07:59:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T07:59:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4 2021-02-20T07:59:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2021-02-20T07:59:40.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8794052555572515) > weight (0.25) 2021-02-20T07:59:40.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.33426562897886436) > weight (0.25) 2021-02-20T07:59:40.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4467389492386794) > weight (0.25) 2021-02-20T07:59:40.450 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T07:59:40.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T07:59:40.451 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T07:59:40.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T07:59:40.454+0000 7fb5f21e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 18999) UID: 1000 2021-02-20T07:59:43.452 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T07:59:43.452 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:00:04.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:00:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:00:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:04.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:04.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:00:04.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:00:04.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:00:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:00:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:05.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:05.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:05.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:00:05.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:00:05.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:00:05.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:00:05.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:00:05.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:00:05.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state+ 2021-02-20T08:00:05.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:local cluster=cluster1 2021-02-20T08:00:05.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:00:05.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:00:05.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:00:05.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:00:05.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:00:05.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:00:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:00:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:00:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:00:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:00:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:00:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:00:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:00:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:00:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:00:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:00:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:00:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:00:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:00:05.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:00:05.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:00:05.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:00:05.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:05.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:00:05.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:00:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:00:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:00:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:00:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:00:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:00:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:00:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:00:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:00:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:00:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:00:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:00:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:00:05.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:00:05.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:00:05.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:00:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:05.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:00:05.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:00:05.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:00:05.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:00:05.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:00:05.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:00:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:00:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:00:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:00:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:00:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:00:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:00:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:00:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:00:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:00:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:00:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:00:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:00:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:00:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:00:05.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:05.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:00:06.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 18 2021-02-20T08:00:06.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:00:06.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:06.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:06.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:06.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:00:06.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:00:06.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:06.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:00:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=18 2021-02-20T08:00:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:00:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:00:07.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:07.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:07.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:07.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:07.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:07.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:07.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:07.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=14 2021-02-20T08:00:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 14 2021-02-20T08:00:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:00:07.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:07.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:00:07.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:07.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:00:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:00:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:00:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:00:07.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 2021-02-20T08:00:07.621 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:00:08.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:08.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:08.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:08.423 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:08.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:08.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:08.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:08.491 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:00:08.491 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:00:08.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:00:08.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:00:08.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:00:08.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 2021-02-20T08:00:08.497 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:00:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:10.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:00:10.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:10.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:00:10.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:00:10.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:00:10.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:00:10.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 2021-02-20T08:00:10.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:00:12.528 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:12.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:00:12.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:12.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:00:12.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:00:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:00:12.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:00:12.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 2021-02-20T08:00:12.597 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:00:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:00:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:00:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:00:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:00:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=18 2021-02-20T08:00:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 18 = 18 2021-02-20T08:00:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2021-02-20T08:00:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:14.672 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:14.672 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:14.673 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4 2021-02-20T08:00:14.673 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:00:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:00:15.671 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2021-02-20T08:00:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:00:15.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:00:15.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:00:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 8 2021-02-20T08:00:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:00:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:00:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:00:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:00:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:00:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:00:15.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:00:15.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 07:59:16 2021 2021-02-20T08:00:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:00:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:00:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:00:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:00:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:00:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:00:15.748 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap3-clean ] 2021-02-20T08:00:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T08:00:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3-clean 2021-02-20T08:00:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:00:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:00:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:00:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:00:16.610 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:17.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:00:17.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:17.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:17.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:17.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T08:00:17.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4 2021-02-20T08:00:17.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:00:17.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:00:17.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:17.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:17.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:00:18.201 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:00:18.202+0000 7f802eca0700 -1 bdev(0x55ac9e4aa000 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 1 2021-02-20T08:00:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:00:18.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:00:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:20.060 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:00:20.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:20.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap4-clean 2021-02-20T08:00:20.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap4-clean 2021-02-20T08:00:20.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4-clean 2021-02-20T08:00:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:00:20.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:00:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:00:20.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:00:20.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:00:20.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:00:22.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 38242304 bytes 2021-02-20T08:00:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 56848384 bytes 2021-02-20T08:00:22.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 70272000 bytes 2021-02-20T08:00:23.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 80010240 bytes 2021-02-20T08:00:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 122263552 bytes 2021-02-20T08:00:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 147072000 bytes 2021-02-20T08:00:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 171061248 bytes 2021-02-20T08:00:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 252571648 bytes 2021-02-20T08:00:25.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 298892288 bytes 2021-02-20T08:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 299740160 bytes 2021-02-20T08:00:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 356795392 bytes 2021-02-20T08:00:25.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 362151936 bytes 2021-02-20T08:00:25.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 372838400 bytes 2021-02-20T08:00:26.439 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 464565248 bytes 2021-02-20T08:00:26.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 521615360 bytes 2021-02-20T08:00:27.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 559081472 bytes 2021-02-20T08:00:27.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 646980608 bytes 2021-02-20T08:00:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 694795264 bytes 2021-02-20T08:00:28.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 727728128 bytes 2021-02-20T08:00:29.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 811977728 bytes 2021-02-20T08:00:29.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 868274176 bytes 2021-02-20T08:00:30.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 966877184 bytes 2021-02-20T08:00:30.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 971513856 bytes 2021-02-20T08:00:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1046779904 bytes 2021-02-20T08:00:31.113 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:00:31.114+0000 7f803c4bb700 -1 reset not still connected to 0x55aca6c592b0 2021-02-20T08:00:31.113 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:00:31.114+0000 7f803c4bb700 -1 reset not still connected to 0x55aca6c5b4d0 2021-02-20T08:00:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1109506048 bytes 2021-02-20T08:00:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 1146718208 bytes 2021-02-20T08:00:32.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2700 IOs, 1223485440 bytes 2021-02-20T08:00:32.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2800 IOs, 1234328576 bytes 2021-02-20T08:00:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:00:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:00:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:00:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:00:32.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:32.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:00:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:00:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:00:32.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:00:33.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:00:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:00:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:00:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:00:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:00:33.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:00:33.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:00:33.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:00:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:00:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":404473856.0,"local_snapshot_timestamp":1613807948,"remote_snapshot_timestamp":1613807948,"replay_state":"idle"} 2021-02-20T08:00:33.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T08:00:33.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:00:05 2021-02-20T08:00:33.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:00:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:00:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:00:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:00:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:00:05 2021-02-20T08:00:33.753 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.754 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:00:33.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":404473856.0,"local_snapshot_timestamp":1613807948,"remote_snapshot_timestamp":1613807948,"replay_state":"idle"} 2021-02-20T08:00:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:00:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T08:00:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:00:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:00:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:00:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:00:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:00:05 2021-02-20T08:00:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:00:05 2021-02-20T08:00:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:00:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:00:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:00:05 2021-02-20T08:00:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:00:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:00:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:00:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:00:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:00:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:00:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:00:33.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:00:33.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:00:05 2021-02-20T08:00:33.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:00:33.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:00:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:00:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T08:00:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:00:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap5 2021-02-20T08:00:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap5 2021-02-20T08:00:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:00:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:00:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:00:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5 2021-02-20T08:00:33.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:00:33.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:00:36.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9423262527732716) > weight (0.25) 2021-02-20T08:00:36.568 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5952712871006888) > weight (0.25) 2021-02-20T08:00:36.568 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44791046439237137) > weight (0.25) 2021-02-20T08:00:36.568 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2860361581221518) > weight (0.25) 2021-02-20T08:00:36.569 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T08:01:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:01:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:01:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:02.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:02.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:02.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:01:02.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:01:02.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:01:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:01:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:01:03.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:01:03.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:01:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:01:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local+ cluster_instance=cluster1 2021-02-20T08:01:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:01:03.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i+ statelocal 2021-02-20T08:01:03.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: cluster=cluster1 2021-02-20T08:01:03.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:01:03.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:01:03.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:01:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:01:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:01:03.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:01:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:01:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:01:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:01:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:01:03.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:01:03.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:01:03.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:01:03.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:03.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:03.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:01:03.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:01:03.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:03.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:01:03.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:01:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:01:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:01:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:01:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:01:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:01:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:01:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:01:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:01:03.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:01:03.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:01:03.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:03.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:01:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:01:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:01:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:01:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:01:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:01:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:01:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:01:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:01:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:01:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:01:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:01:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:01:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:01:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:01:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:01:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:01:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:01:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:01:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:03.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:03.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:03.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:01:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 22 2021-02-20T08:01:04.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:04.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:04.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:04.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:01:04.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:04.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:01:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=22 2021-02-20T08:01:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:01:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:05.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:05.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:05.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:05.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:05.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:05.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=18 2021-02-20T08:01:05.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 18 2021-02-20T08:01:05.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:01:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:05.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:05.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:05.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 2021-02-20T08:01:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:01:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:01:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:06.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 2021-02-20T08:01:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:01:08.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:08.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:08.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:08.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:08.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:08.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:08.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:08.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:08.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 2021-02-20T08:01:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:01:08.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:01:08.883 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:01:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:01:08.884 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:01:08.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:01:08.887 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T08:01:10.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:10.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:10.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:10.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:10.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:10.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:10.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 2021-02-20T08:01:10.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:01:12.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:12.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:12.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=22 2021-02-20T08:01:12.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 22 = 22 2021-02-20T08:01:12.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:12.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2021-02-20T08:01:12.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:12.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:12.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:12.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5 2021-02-20T08:01:12.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:01:12.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:01:13.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2021-02-20T08:01:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:01:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:01:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:01:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 10 2021-02-20T08:01:13.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:01:13.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:01:13.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:01:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:01:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:01:13.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:01:13.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:01:13.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:00:17 2021 2021-02-20T08:01:13.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:01:13.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:01:13.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:01:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:01:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:01:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:01:13.528 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:13.528 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap4-clean ] 2021-02-20T08:01:13.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T08:01:13.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:13.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:13.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4-clean 2021-02-20T08:01:13.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:01:13.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:01:13.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:13.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:13.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:01:13.957 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:01:13.953+0000 7f802a497700 -1 bdev(0x55ac9e4aa000 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 1 2021-02-20T08:01:14.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:01:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:01:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:15.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T08:01:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5 2021-02-20T08:01:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:01:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:01:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:01:16.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:01:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:17.566 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:01:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:01:17.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:17.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap5-clean 2021-02-20T08:01:17.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap5-clean 2021-02-20T08:01:17.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:17.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:17.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:17.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5-clean 2021-02-20T08:01:17.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:01:18.764 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:01:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:01:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=7 2021-02-20T08:01:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:01:18.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:01:19.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 36593664 bytes 2021-02-20T08:01:20.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 119229440 bytes 2021-02-20T08:01:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 142399488 bytes 2021-02-20T08:01:21.630 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 162336768 bytes 2021-02-20T08:01:21.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 164410368 bytes 2021-02-20T08:01:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 286644224 bytes 2021-02-20T08:01:22.949 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 312834048 bytes 2021-02-20T08:01:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 313450496 bytes 2021-02-20T08:01:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 363885568 bytes 2021-02-20T08:01:23.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 412605440 bytes 2021-02-20T08:01:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 455711744 bytes 2021-02-20T08:01:24.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 474530816 bytes 2021-02-20T08:01:24.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 529399808 bytes 2021-02-20T08:01:25.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 552300544 bytes 2021-02-20T08:01:25.771 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:01:25.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:01:25.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:01:25.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:01:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:01:25.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:01:25.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:01:25.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:01:26.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:01:26.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:01:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:01:26.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:01:26.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:26.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:01:26.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:01:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:01:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:01:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":487463936.0,"local_snapshot_timestamp":1613808006,"remote_snapshot_timestamp":1613808006,"replay_state":"idle"} 2021-02-20T08:01:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T08:01:26.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:01:05 2021-02-20T08:01:26.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:01:26.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:01:26.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:01:26.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:01:26.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:01:05 2021-02-20T08:01:26.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:01:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":487463936.0,"local_snapshot_timestamp":1613808006,"remote_snapshot_timestamp":1613808006,"replay_state":"idle"} 2021-02-20T08:01:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:01:26.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:01:26.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:01:26.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:01:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:01:26.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T08:01:26.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:01:05 2021-02-20T08:01:26.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:01:05 2021-02-20T08:01:26.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:26.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:26.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:01:26.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:01:05 2021-02-20T08:01:26.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:01:26.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:01:26.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:26.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:01:26.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:01:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:01:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:01:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:01:26.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:01:05 2021-02-20T08:01:26.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:01:27.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:01:27.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:01:27.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T08:01:27.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:01:27.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:27.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:27.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap6 2021-02-20T08:01:27.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap6 2021-02-20T08:01:27.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:27.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:27.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:27.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6 2021-02-20T08:01:27.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:01:40.570 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2503494025896932) > weight (0.25) 2021-02-20T08:01:40.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7516303604511047) > weight (0.25) 2021-02-20T08:01:40.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4354501312982534) > weight (0.25) 2021-02-20T08:01:40.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2895281907355346) > weight (0.25) 2021-02-20T08:01:40.572 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T08:01:55.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:01:55.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:55.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:55.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:55.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:01:55.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:55.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:55.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:55.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:01:55.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:01:55.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:01:56.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:01:56.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:56.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:56.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:56.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:01:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:01:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:01:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:01:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:01:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:01:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1grep 2021-02-20T08:01:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: -i state 2021-02-20T08:01:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:01:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:01:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:01:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:01:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:01:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:01:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:01:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:01:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:01:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:01:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:01:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:01:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:01:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:01:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:01:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:01:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:01:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:01:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:01:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:01:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:01:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:01:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:01:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:01:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:01:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:01:56.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:56.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:01:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:01:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:01:56.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:01:56.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:01:56.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:01:56.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:01:56.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:01:56.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:01:56.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:01:56.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:56.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:56.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:01:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:01:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:01:56.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:01:56.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:56.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:01:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:01:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:56.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:01:56.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:56.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:01:56.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:01:56.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:01:56.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:56.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:01:56.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:56.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:56.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:56.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:01:57.823 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 26 2021-02-20T08:01:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:01:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:01:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:01:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:57.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:01:57.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=26 2021-02-20T08:01:57.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:57.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:01:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:58.187 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:58.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=22 2021-02-20T08:01:58.192 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 22 2021-02-20T08:01:58.192 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:01:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 2021-02-20T08:01:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:01:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:01:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:01:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:01:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:01:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:01:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:01:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:01:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:01:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:01:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:01:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:01:59.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:01:59.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 2021-02-20T08:01:59.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:02:01.152 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:01.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:01.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:01.341 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 2021-02-20T08:02:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:02:01.762 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T08:02:01.757+0000 7f62c7ffc700 -1 reset not still connected to 0x555b2f70c410 2021-02-20T08:02:03.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:03.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:03.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:03.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:03.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:03.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:03.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:03.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:03.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:03.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:03.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:03.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:03.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:03.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 2021-02-20T08:02:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:02:05.427 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:05.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:05.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:05.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:05.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=26 2021-02-20T08:02:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 26 = 26 2021-02-20T08:02:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2021-02-20T08:02:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:05.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6 2021-02-20T08:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:02:06.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2021-02-20T08:02:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:02:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:02:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:02:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 12 2021-02-20T08:02:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:02:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:02:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:02:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:02:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:02:06.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:02:06.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:02:06.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:00:17 2021 2021-02-20T08:02:06.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:02:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:02:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:02:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:02:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:02:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:02:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:06.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap5-clean ] 2021-02-20T08:02:06.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T08:02:06.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5-clean 2021-02-20T08:02:06.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:02:06.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:02:06.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:06.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:02:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:02:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:08.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:02:08.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:08.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T08:02:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6 2021-02-20T08:02:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:02:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:02:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:02:09.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:02:09.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:10.442 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:10.432+0000 7f1c68dc9700 -1 bdev(0x555a508f6000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T08:02:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:02:10.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:11.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:02:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap6-clean 2021-02-20T08:02:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap6-clean 2021-02-20T08:02:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6-clean 2021-02-20T08:02:11.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:02:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:02:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:11.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:11.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:11.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:02:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=10 2021-02-20T08:02:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:02:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:02:13.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 26893312 bytes 2021-02-20T08:02:13.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 82641920 bytes 2021-02-20T08:02:14.600 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 178990080 bytes 2021-02-20T08:02:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 224489472 bytes 2021-02-20T08:02:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 253521920 bytes 2021-02-20T08:02:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 324979712 bytes 2021-02-20T08:02:16.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 359443456 bytes 2021-02-20T08:02:16.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 399916032 bytes 2021-02-20T08:02:17.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 429152256 bytes 2021-02-20T08:02:17.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 448349184 bytes 2021-02-20T08:02:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 497161216 bytes 2021-02-20T08:02:17.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 497777664 bytes 2021-02-20T08:02:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 530222080 bytes 2021-02-20T08:02:18.430 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 549599232 bytes 2021-02-20T08:02:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 559849472 bytes 2021-02-20T08:02:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 641429504 bytes 2021-02-20T08:02:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 687456256 bytes 2021-02-20T08:02:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 720040960 bytes 2021-02-20T08:02:20.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 756035584 bytes 2021-02-20T08:02:20.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 795844608 bytes 2021-02-20T08:02:21.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 830840832 bytes 2021-02-20T08:02:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 845708288 bytes 2021-02-20T08:02:21.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 846322688 bytes 2021-02-20T08:02:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 881657856 bytes 2021-02-20T08:02:21.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:02:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:02:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:02:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:02:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:02:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:02:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:02:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:02:22.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:02:22.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:22.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:22.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:22.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:02:22.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:02:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:02:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:02:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd+ --cluster cluster1 mirror image status mirror/test 2021-02-20T08:02:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:02:22.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":12666094.93,"bytes_per_snapshot":434479616.0,"local_snapshot_timestamp":1613808117,"remote_snapshot_timestamp":1613808117,"replay_state":"idle"} 2021-02-20T08:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T08:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:02:05 2021-02-20T08:02:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:02:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:02:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:02:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:02:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:02:05 2021-02-20T08:02:22.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:02:22.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":12666094.93,"bytes_per_snapshot":434479616.0,"local_snapshot_timestamp":1613808117,"remote_snapshot_timestamp":1613808117,"replay_state":"idle"} 2021-02-20T08:02:22.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:02:22.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:02:22.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:02:22.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:02:22.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T08:02:22.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:02:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:02:05 2021-02-20T08:02:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:02:05 2021-02-20T08:02:22.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:02:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:02:05 2021-02-20T08:02:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:02:22.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:02:22.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:02:22.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:02:22.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:02:22.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:02:23.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:02:23.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:02:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:02:05 2021-02-20T08:02:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:02:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:02:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:02:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T08:02:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:02:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap7 2021-02-20T08:02:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap7 2021-02-20T08:02:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:23.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7 2021-02-20T08:02:23.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:02:47.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3202465060615759) > weight (0.25) 2021-02-20T08:02:47.892 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45282574574711276) > weight (0.25) 2021-02-20T08:02:47.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3170082078344566) > weight (0.25) 2021-02-20T08:02:47.895 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5126217785329076) > weight (0.25) 2021-02-20T08:02:47.896 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T08:02:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:02:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:02:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:02:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:02:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:02:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:02:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:02:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:02:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:02:52.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:02:52.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T08:02:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:02:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local cluster=cluster1 2021-02-20T08:02:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:02:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:02:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T08:02:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:02:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:02:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:02:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:02:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:02:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:02:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:02:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:02:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:02:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:02:52.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:02:52.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:02:52.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:02:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:02:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:02:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:02:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:02:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:02:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:02:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:02:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:02:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:02:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:02:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:02:52.850 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:02:52.850 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:02:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:02:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:02:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:02:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:02:52.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:02:52.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:02:52.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:02:52.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:02:52.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:02:52.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:02:52.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:02:52.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:02:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:02:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:02:52.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:02:52.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:02:52.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:02:52.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:02:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:02:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:02:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:02:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:02:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:02:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:02:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:02:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:02:52.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:02:52.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:02:52.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:02:52.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:02:52.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:02:52.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:52.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:02:52.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:52.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:52.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:52.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:02:53.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 30 2021-02-20T08:02:53.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:02:53.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:02:53.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:53.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:53.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:02:53.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:02:53.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:53.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:02:53.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=30 2021-02-20T08:02:53.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:53.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:02:54.156 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:54.156 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:54.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:54.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=26 2021-02-20T08:02:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 26 2021-02-20T08:02:54.242 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:02:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:54.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:54.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:54.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:54.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:54.709 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:54.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:54.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:54.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:54.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:54.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:54.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 2021-02-20T08:02:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:02:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:55.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:55.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:55.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:55.598 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:55.598 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:55.603 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:55.603 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:55.603 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 2021-02-20T08:02:55.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:02:57.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:57.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:57.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:57.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:57.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:57.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:57.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:57.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:57.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:57.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:57.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:57.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:57.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:57.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 2021-02-20T08:02:57.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:02:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:02:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:02:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:02:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:02:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:02:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:02:59.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:02:59.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:02:59.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:02:59.348 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:02:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:02:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:02:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 2021-02-20T08:02:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:02:59.397 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:59.390+0000 7f1c785e8700 -1 reset not still connected to 0x555a51837520 2021-02-20T08:02:59.398 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:59.390+0000 7f1c785e8700 -1 reset not still connected to 0x555a51837c70 2021-02-20T08:02:59.399 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:59.390+0000 7f1c785e8700 -1 reset not still connected to 0x555a51838e50 2021-02-20T08:02:59.399 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:59.390+0000 7f1c785e8700 -1 reset not still connected to 0x555a51838f20 2021-02-20T08:02:59.400 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:02:59.390+0000 7f1c785e8700 -1 reset not still connected to 0x555a51839810 2021-02-20T08:03:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:01.430 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=30 2021-02-20T08:03:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 30 = 30 2021-02-20T08:03:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2021-02-20T08:03:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7 2021-02-20T08:03:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:03:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:03:02.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2021-02-20T08:03:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:03:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:03:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:03:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 14 2021-02-20T08:03:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:03:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:03:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:03:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:03:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:03:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:03:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:03:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:02:07 2021 2021-02-20T08:03:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:03:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:03:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:03:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:03:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:03:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:03:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap6-clean ] 2021-02-20T08:03:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T08:03:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6-clean 2021-02-20T08:03:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:03:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:03.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:03.341 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:04.425 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:04.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:04.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T08:03:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7 2021-02-20T08:03:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:03:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:04.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:05.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:06.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:06.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:07.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap7-clean 2021-02-20T08:03:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap7-clean 2021-02-20T08:03:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7-clean 2021-02-20T08:03:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:03:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:03:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:07.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:07.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:03:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:03:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:03:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:03:08.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 6155264 bytes 2021-02-20T08:03:09.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 84511744 bytes 2021-02-20T08:03:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 160976896 bytes 2021-02-20T08:03:11.155 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 203924480 bytes 2021-02-20T08:03:11.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 283073536 bytes 2021-02-20T08:03:12.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 301614080 bytes 2021-02-20T08:03:12.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 402394112 bytes 2021-02-20T08:03:13.398 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 465566720 bytes 2021-02-20T08:03:13.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 466165760 bytes 2021-02-20T08:03:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 500959232 bytes 2021-02-20T08:03:14.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 553933824 bytes 2021-02-20T08:03:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 573914112 bytes 2021-02-20T08:03:15.230 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 626205696 bytes 2021-02-20T08:03:15.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:03:15.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:03:15.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:03:15.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:15.711 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:03:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:03:15.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:03:15.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:03:15.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:03:16.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:03:16.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:16.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:16.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:16.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:03:16.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:03:16.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:03:16.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.714 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:03:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:03:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:03:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:03:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:03:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":14562918.4,"bytes_per_snapshot":408435200.0,"local_snapshot_timestamp":1613808173,"remote_snapshot_timestamp":1613808173,"replay_state":"idle"} 2021-02-20T08:03:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T08:03:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:03:05 2021-02-20T08:03:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:03:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:03:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:03:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:03:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:03:05 2021-02-20T08:03:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:03:16.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":14562918.4,"bytes_per_snapshot":408435200.0,"local_snapshot_timestamp":1613808173,"remote_snapshot_timestamp":1613808173,"replay_state":"idle"} 2021-02-20T08:03:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:03:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:03:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:03:16.808 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:03:16.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T08:03:16.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:03:16.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:03:05 2021-02-20T08:03:16.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:03:05 2021-02-20T08:03:16.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:16.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:16.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:16.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:03:16.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:03:05 2021-02-20T08:03:16.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:03:16.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:03:16.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:03:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:03:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:03:16.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:03:16.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:03:16.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:03:16.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:03:05 2021-02-20T08:03:16.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:03:16.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:03:16.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + echo up+replaying 2021-02-20T08:03:16.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep up+replaying 2021-02-20T08:03:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:03:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap8 2021-02-20T08:03:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap8 2021-02-20T08:03:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:16.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:16.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:16.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8 2021-02-20T08:03:16.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:03:38.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3492206723217035) > weight (0.25) 2021-02-20T08:03:38.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8127744894295151) > weight (0.25) 2021-02-20T08:03:38.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25875817735234463) > weight (0.25) 2021-02-20T08:03:38.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2710853197479647) > weight (0.25) 2021-02-20T08:03:38.574 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T08:03:43.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:03:43.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:43.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:43.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:43.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:03:43.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:03:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:03:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:03:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:03:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:03:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:03:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:03:44.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:03:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:03:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T08:03:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:03:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:03:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:03:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:03:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:03:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:03:44.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:03:44.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:03:44.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:03:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:03:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:03:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:03:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:03:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:03:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:03:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:03:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:03:44.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:03:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:03:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:03:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:03:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:03:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:03:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:03:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:03:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:03:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:03:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:03:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:03:44.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:03:44.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:03:44.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:03:44.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:03:44.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:03:44.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:03:44.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:03:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:03:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:45.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:03:45.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:03:45.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:03:45.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:03:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:03:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:03:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:03:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:03:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:03:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:03:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:03:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:03:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:03:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:03:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:03:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:03:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:03:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:03:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:03:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:03:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:03:45.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 34 2021-02-20T08:03:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:03:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:03:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:03:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:03:45.957 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=34 2021-02-20T08:03:45.957 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:03:45.957 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:03:46.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:46.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:46.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:46.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:46.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:46.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:46.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:46.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:46.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=30 2021-02-20T08:03:46.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 30 2021-02-20T08:03:46.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:03:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:46.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=30 2021-02-20T08:03:46.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 30 2021-02-20T08:03:46.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:03:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:47.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:47.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:47.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:47.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:03:47.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:47.608 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:03:47.608 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:03:47.609 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:03:47.609 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:03:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 2021-02-20T08:03:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:03:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:49.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:03:49.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:49.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:03:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:03:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:03:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:03:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 2021-02-20T08:03:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:03:51.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:03:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:51.376 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:03:51.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:03:51.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:03:51.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:03:51.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 2021-02-20T08:03:51.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:03:53.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:53.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:53.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:53.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:53.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:03:53.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:03:53.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:03:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:03:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=34 2021-02-20T08:03:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 34 = 34 2021-02-20T08:03:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2021-02-20T08:03:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:03:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8 2021-02-20T08:03:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:03:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:03:54.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2021-02-20T08:03:54.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:03:54.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:03:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:03:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 16 2021-02-20T08:03:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:03:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:03:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:03:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:03:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:03:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:03:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:03:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:02:07 2021 2021-02-20T08:03:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:03:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:03:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:03:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:03:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:03:54.539 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:03:54.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:54.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap7-clean ] 2021-02-20T08:03:54.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T08:03:54.541 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:54.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:54.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7-clean 2021-02-20T08:03:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:03:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:54.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:55.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:55.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:56.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T08:03:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:56.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:56.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8 2021-02-20T08:03:56.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:03:56.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:56.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:56.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:56.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:03:56.950 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:03:56.942+0000 7ff4e9ca2700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 5 2021-02-20T08:03:57.291 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:03:57.286+0000 7ff4ecca8700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T08:03:57.291 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:03:57.286+0000 7ff4e749d700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T08:03:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:57.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:03:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:03:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:03:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap8-clean 2021-02-20T08:03:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap8-clean 2021-02-20T08:03:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8-clean 2021-02-20T08:03:59.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:03:59.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:03:59.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:03:59.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:03:59.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:03:59.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:03:59.855 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:03:59.855 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:03:59.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:04:01.634 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 84785152 bytes 2021-02-20T08:04:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 143987712 bytes 2021-02-20T08:04:02.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 158152704 bytes 2021-02-20T08:04:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 234767360 bytes 2021-02-20T08:04:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 266082304 bytes 2021-02-20T08:04:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 362978304 bytes 2021-02-20T08:04:04.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 409945088 bytes 2021-02-20T08:04:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 467774464 bytes 2021-02-20T08:04:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 532082688 bytes 2021-02-20T08:04:06.403 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 567366656 bytes 2021-02-20T08:04:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 652628992 bytes 2021-02-20T08:04:07.294 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 684792832 bytes 2021-02-20T08:04:07.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 712295424 bytes 2021-02-20T08:04:07.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 723795968 bytes 2021-02-20T08:04:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 729342976 bytes 2021-02-20T08:04:08.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 754042880 bytes 2021-02-20T08:04:08.667 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 786902016 bytes 2021-02-20T08:04:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 816211968 bytes 2021-02-20T08:04:09.462 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 873736192 bytes 2021-02-20T08:04:10.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 944660480 bytes 2021-02-20T08:04:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 1051898880 bytes 2021-02-20T08:04:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1146639360 bytes 2021-02-20T08:04:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:04:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:04:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:04:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:04:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:04:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:04:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:04:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:04:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:04:12.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:04:12.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:04:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:04:12.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:04:12.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:04:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:04:12.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:04:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:04:12.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:04:12.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":13318792.53,"bytes_per_snapshot":418225664.0,"local_snapshot_timestamp":1613808225,"remote_snapshot_timestamp":1613808225,"replay_state":"idle"} 2021-02-20T08:04:12.951 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.0 on smithi114 2021-02-20T08:04:12.951 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:04:05 2021-02-20T08:04:12.952 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:04:12.952 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:04:12.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:04:12.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:04:12.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:04:05 2021-02-20T08:04:12.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:04:12.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":13318792.53,"bytes_per_snapshot":418225664.0,"local_snapshot_timestamp":1613808225,"remote_snapshot_timestamp":1613808225,"replay_state":"idle"} 2021-02-20T08:04:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:04:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:04:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:04:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:04:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.0 on smithi114 2021-02-20T08:04:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:04:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:04:05 2021-02-20T08:04:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:04:05 2021-02-20T08:04:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:04:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:04:05 2021-02-20T08:04:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:04:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:04:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:04:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:04:12.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:04:13.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:04:13.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:04:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:04:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:04:05 2021-02-20T08:04:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:04:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:04:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:04:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep up+replaying 2021-02-20T08:04:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:04:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:04:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:04:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap9 2021-02-20T08:04:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap9 2021-02-20T08:04:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:04:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9 2021-02-20T08:04:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:04:41.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:04:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:04:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:41.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:04:41.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:04:41.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:04:42.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:04:42.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:42.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:42.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:42.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:04:42.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:04:42.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:04:42.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:04:42.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + local cluster_instance=cluster1 2021-02-20T08:04:42.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T08:04:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:04:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:04:42.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:04:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:04:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:04:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:04:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:04:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:04:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:42.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:42.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:42.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:04:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:04:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:42.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:42.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:04:42.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:04:42.907 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:04:42.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:04:42.908 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:04:42.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:04:42.906+0000 7f733b7ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 18992) UID: 1000 2021-02-20T08:04:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:42.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:04:43.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:04:43.022+0000 7f733bfef700 -1 asok(0x564525a42000) AdminSocket: request '[{"prefix": "rbd mirror status mirror/test"}]' not defined 2021-02-20T08:04:43.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:04:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= 2021-02-20T08:04:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 1 2021-02-20T08:04:43.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:04:43.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:04:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:04:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:04:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:04:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:04:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:04:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:04:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:04:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T08:04:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local+ cluster=cluster1 2021-02-20T08:04:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:04:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:04:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep -i state 2021-02-20T08:04:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:04:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:04:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:04:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:04:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:04:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:04:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:04:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:04:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:04:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:04:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:04:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2021-02-20T08:04:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:04:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:04:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:04:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:04:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:04:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:04:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:04:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:04:45.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:45.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:45.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:04:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:04:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:04:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:45.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:04:45.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:04:45.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:45.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:04:45.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:45.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:45.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:45.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:45.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:45.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:45.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:45.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:45.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:04:45.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:45.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:45.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:45.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:04:45.403 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:45.404 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:45.404 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:45.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:04:45.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:04:45.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:45.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:04:45.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:45.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:45.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.408 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:45.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:45.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:45.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:45.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:45.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:04:45.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:04:45.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:45.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:45.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:04:45.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:45.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:45.411 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:45.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:45.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:45.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:45.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:45.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:45.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:45.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:45.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:04:45.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:45.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:45.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:45.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:04:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:45.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:45.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:45.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:04:46.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:04:46.531 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:04:46.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:04:46.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:04:46.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:46.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:04:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:04:46.537 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:04:46.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:46.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:04:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:46.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:46.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:04:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:04:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2021-02-20T08:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:46.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:46.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:46.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:46.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:46.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:46.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:04:46.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:46.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:04:46.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:46.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:04:46.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:04:46.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:46.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:04:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:46.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:04:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:04:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:04:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:46.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:46.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:46.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:46.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:46.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:46.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:04:46.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:04:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:46.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:04:46.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:04:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:04:46.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:46.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:46.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:46.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:46.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:46.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:04:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:04:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:46.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:04:46.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:46.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:46.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:46.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:04:46.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:46.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:46.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:46.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:04:47.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:47.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:47.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:47.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:04:48.909 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:04:48.910 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:04:48.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:04:48.910 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:04:48.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:04:48.911 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:04:48.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:04:48.914 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T08:04:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:04:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:04:49.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:04:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:04:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:04:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:49.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:49.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:04:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:04:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:04:49.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:49.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:49.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:04:49.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:49.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:49.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:49.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:04:49.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:04:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:04:49.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:49.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:04:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:49.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:49.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:49.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:49.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:49.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:04:49.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:04:49.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:49.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:04:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:49.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:49.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:49.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:49.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:49.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:49.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:04:49.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:49.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:49.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:49.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:04:49.263 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:04:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:04:49.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:04:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:49.267 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:49.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:49.268 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:49.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:49.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:49.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:04:49.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:04:49.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:49.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:04:49.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:49.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:49.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:49.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:49.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:04:49.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:49.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:49.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:49.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:04:49.381 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:49.381 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:49.382 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:49.383 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:04:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:04:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:04:49.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:49.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:49.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:04:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:04:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:04:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:49.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:49.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:49.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:49.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:04:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:49.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:49.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:49.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:04:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 4 2021-02-20T08:04:53.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:04:53.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:04:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:04:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:04:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:04:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:04:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:04:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:04:53.516 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:04:53.516 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:53.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:04:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:04:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:04:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:04:53.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:53.526 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:53.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:04:53.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:04:53.635 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:53.635 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:53.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:04:53.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:04:53.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:04:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:53.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:53.640 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:04:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:04:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:04:53.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:04:53.643 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:53.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:53.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:04:53.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:04:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:04:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:04:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:04:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:04:53.752 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:53.753 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:53.753 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:53.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:04:53.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:04:53.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:53.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:04:53.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:53.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:53.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:53.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:53.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:53.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:53.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:53.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:04:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:04:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:04:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:04:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:53.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:04:53.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:53.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:04:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:04:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:04:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:04:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:04:53.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:53.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:53.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:04:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:04:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:04:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:04:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:04:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:04:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:04:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:04:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:04:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:04:53.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:04:53.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:04:53.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:04:53.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:04:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:04:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:04:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:04:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:04:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:04:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:04:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:04:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:04:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:04:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:04:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:04:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:04:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T08:05:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:05:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:05:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:05:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:05:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:05:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:05:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:05:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:05:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:01.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:01.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:01.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:01.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:05:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:05:02.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:02.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:02.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:02.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:05:02.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:05:02.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:02.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:05:02.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:05:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:05:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:05:02.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:02.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:05:02.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:02.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:02.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:02.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:05:02.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:02.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:02.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:05:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:05:02.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:02.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:05:02.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:05:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:05:02.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:05:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:02.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:05:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:05:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:02.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:02.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:05:02.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:05:02.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:02.358 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:05:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:05:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:05:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:05:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:05:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:05:02.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:02.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:02.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:05:02.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:05:02.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:02.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:02.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:02.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:05:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:05:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:02.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:05:02.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:02.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:02.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:02.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:05:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:02.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:02.476 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:02.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T08:05:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:05:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:05:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:05:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:05:10.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:05:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:10.481 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:05:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:05:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:05:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:05:10.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:10.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:10.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:10.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:05:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:10.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:05:10.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:05:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:05:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:10.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:10.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:05:10.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:05:10.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:10.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:05:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:10.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:10.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:10.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:10.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:10.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:10.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:05:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:05:10.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:10.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:10.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:10.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:05:10.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:05:10.739 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:10.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:05:10.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:10.741 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:10.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:10.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:10.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:10.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.744 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:10.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:10.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:05:10.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:05:10.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:10.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:05:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:10.749 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:10.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:10.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:05:10.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:10.759 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:05:10.858 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:10.860 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-02-20T08:05:10.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:3 2021-02-20T08:05:10.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:10.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-02-20T08:05:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:05:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:05:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:05:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-02-20T08:05:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:3 2021-02-20T08:05:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-02-20T08:05:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:3 2021-02-20T08:05:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 3 = cluster1:3 ] 2021-02-20T08:05:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=3 2021-02-20T08:05:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=3 2021-02-20T08:05:10.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-02-20T08:05:10.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:10.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:10.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-02-20T08:05:10.877 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-02-20T08:05:10.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:10.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:10.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:10.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T08:05:18.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:05:18.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:05:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:05:18.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:05:18.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:18.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:18.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:05:18.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:18.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:18.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:18.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:18.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:18.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:18.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:18.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:18.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:05:18.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:05:18.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:18.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:18.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:05:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:05:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:18.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:18.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:18.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:18.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:05:18.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:18.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:18.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:05:18.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:05:18.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:05:18.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:18.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:18.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:05:18.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:05:19.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:19.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:05:19.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:05:19.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:19.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:19.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:05:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:05:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:05:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:19.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:05:19.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:05:19.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:19.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:19.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:19.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:05:19.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:19.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:19.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:05:19.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:05:19.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:05:19.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:19.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:19.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:05:19.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:05:19.229 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:05:19.229 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:05:19.230 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:05:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:05:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:05:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:05:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:05:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:05:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:19.232 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:19.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:19.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:05:19.233 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:19.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:19.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:05:19.234 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:05:19.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:19.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:05:19.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:05:19.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:05:19.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:05:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:05:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:05:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:05:19.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:05:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:05:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:05:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:05:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:05:19.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:05:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:05:19.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:19.359 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:05:19.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:05:19.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:05:19.361 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:05:19.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:05:19.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:05:19.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:19.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:05:19.363 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:05:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:05:19.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:05:19.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:05:19.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:05:19.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:19.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:05:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:05:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:05:19.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:05:19.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:05:19.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:05:19.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:05:19.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:19.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:05:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:05:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 38 2021-02-20T08:05:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:05:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:19.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:19.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:19.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:05:19.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:05:19.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:19.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:05:19.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=38 2021-02-20T08:05:19.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:05:19.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:05:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:20.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:20.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:20.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:20.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:20.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:20.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:20.266 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:20.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=34 2021-02-20T08:05:20.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 34 2021-02-20T08:05:20.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:05:20.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5011283847516977) > weight (0.25) 2021-02-20T08:05:20.576 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:05:20.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:05:20.577 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:05:20.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:05:20.573+0000 7f78efee0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 22485) UID: 1000 2021-02-20T08:05:20.674 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:20.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:20.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:20.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:20.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:20.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:20.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:20.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:05:20.745 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:20.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:05:20.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:05:20.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:05:20.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:05:20.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 2021-02-20T08:05:20.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:05:20.778 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:05:21.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:21.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:21.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:21.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:21.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:21.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:21.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:21.632 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:05:21.632 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:05:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:05:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:05:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:05:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 2021-02-20T08:05:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:05:22.578 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:05:22.578 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:05:22.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:05:22.579 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:05:22.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:05:22.580 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:05:22.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:05:22.583 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T08:05:23.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:23.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:23.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:23.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:23.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:23.240 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:05:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:05:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:05:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:05:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:05:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 2021-02-20T08:05:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:05:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:25.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:25.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:25.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:25.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:25.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:25.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:05:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:25.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:05:25.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:05:25.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:05:25.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:05:25.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 2021-02-20T08:05:25.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:05:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:27.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:27.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:27.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:27.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:05:27.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:05:27.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:05:27.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:05:27.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=38 2021-02-20T08:05:27.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 38 = 38 2021-02-20T08:05:27.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:27.502 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2021-02-20T08:05:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9 2021-02-20T08:05:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:05:27.504 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:05:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2021-02-20T08:05:28.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:05:28.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:05:28.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:05:28.575 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 18 2021-02-20T08:05:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:05:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:05:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:05:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:05:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:05:28.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:05:28.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:05:28.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:03:55 2021 2021-02-20T08:05:28.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:05:28.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:05:28.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:05:28.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:05:28.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:05:28.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:05:28.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap8-clean ] 2021-02-20T08:05:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T08:05:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:28.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8-clean 2021-02-20T08:05:28.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:05:28.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:05:28.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:28.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:28.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:05:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:05:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:30.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:05:30.522 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:30.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:30.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T08:05:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9 2021-02-20T08:05:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:05:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:05:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:30.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:30.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:05:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:05:31.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:32.790 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:05:32.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:33.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:05:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap9-clean 2021-02-20T08:05:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap9-clean 2021-02-20T08:05:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9-clean 2021-02-20T08:05:33.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:05:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror test 2021-02-20T08:05:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:05:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-02-20T08:05:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 46729216 bytes 2021-02-20T08:05:35.945 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 50293760 bytes 2021-02-20T08:05:36.498 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 87651328 bytes 2021-02-20T08:05:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 191777792 bytes 2021-02-20T08:05:37.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 236353536 bytes 2021-02-20T08:05:38.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 308589568 bytes 2021-02-20T08:05:38.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 350990336 bytes 2021-02-20T08:05:39.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 377121792 bytes 2021-02-20T08:05:39.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 400803840 bytes 2021-02-20T08:05:39.753 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 447705088 bytes 2021-02-20T08:05:40.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 514850816 bytes 2021-02-20T08:05:40.489 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 525822976 bytes 2021-02-20T08:05:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 549588992 bytes 2021-02-20T08:05:40.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 569339904 bytes 2021-02-20T08:05:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 588093440 bytes 2021-02-20T08:05:41.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 613951488 bytes 2021-02-20T08:05:42.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 676027392 bytes 2021-02-20T08:05:42.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 719694848 bytes 2021-02-20T08:05:42.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 814460928 bytes 2021-02-20T08:05:43.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 826491904 bytes 2021-02-20T08:05:43.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 873072640 bytes 2021-02-20T08:05:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 954245120 bytes 2021-02-20T08:05:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 985108480 bytes 2021-02-20T08:05:45.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1041940480 bytes 2021-02-20T08:05:45.497 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1091290112 bytes 2021-02-20T08:05:45.754 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 1113888768 bytes 2021-02-20T08:05:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:05:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:05:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:05:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:05:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:05:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:05:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:05:45.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:05:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-02-20T08:05:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:05:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local description_pattern= 2021-02-20T08:05:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local service_pattern= 2021-02-20T08:05:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:05:46.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-02-20T08:05:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:test: 2021-02-20T08:05:46.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: global_id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:05:46.970 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+replaying 2021-02-20T08:05:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: replaying, {"bytes_per_second":29080033.88,"bytes_per_snapshot":494360576.0,"local_snapshot_timestamp":1613808319,"remote_snapshot_timestamp":1613808319,"replay_state":"idle"} 2021-02-20T08:05:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: service: mirror.2 on smithi114 2021-02-20T08:05:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:05:35 2021-02-20T08:05:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: peer_sites: 2021-02-20T08:05:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: name: cluster2 2021-02-20T08:05:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: state: up+stopped 2021-02-20T08:05:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: description: local image is primary 2021-02-20T08:05:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: last_update: 2021-02-20 08:05:35 2021-02-20T08:05:46.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: state: up+replaying 2021-02-20T08:05:46.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^ description: .* /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: description: replaying, {"bytes_per_second":29080033.88,"bytes_per_snapshot":494360576.0,"local_snapshot_timestamp":1613808319,"remote_snapshot_timestamp":1613808319,"replay_state":"idle"} 2021-02-20T08:05:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:05:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo up+replaying 2021-02-20T08:05:46.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep ^up+ 2021-02-20T08:05:46.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:05:46.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep service: *mirror..* on /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: service: mirror.2 on smithi114 2021-02-20T08:05:46.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.mirror_status 2021-02-20T08:05:46.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update=2021-02-20 08:05:35 2021-02-20T08:05:46.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-02-20 08:05:35 2021-02-20T08:05:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:05:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state_pattern=up+replaying 2021-02-20T08:05:46.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local prev_last_update=2021-02-20 08:05:35 2021-02-20T08:05:46.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror.mirror_status 2021-02-20T08:05:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror.mirror_status 2021-02-20T08:05:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:05:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror.mirror_status 2021-02-20T08:05:46.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-02-20T08:05:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: failed to query services: (13) Permission denied 2021-02-20T08:05:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local last_update state 2021-02-20T08:05:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-02-20T08:05:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ last_update=2021-02-20 08:05:35 2021-02-20T08:05:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-02-20T08:05:47.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ state=up+replaying 2021-02-20T08:05:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + echo up+replaying 2021-02-20T08:05:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:grep up+replaying 2021-02-20T08:05:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:05:47.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap10 2021-02-20T08:05:47.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap10 2021-02-20T08:05:47.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:05:47.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:05:47.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:05:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10 2021-02-20T08:05:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:05:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:up+replaying 2021-02-20T08:06:17.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:06:17.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:17.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:17.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:17.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:06:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:17.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:17.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:06:17.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:06:17.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:06:18.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:06:18.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:06:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:06:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:06:18.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:06:18.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T08:06:18.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:18.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:06:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loop_instance 2021-02-20T08:06:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:06:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i state 2021-02-20T08:06:18.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:06:18.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:06:18.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:06:18.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:06:18.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:18.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:18.798 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:06:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:06:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:06:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:06:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:06:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:06:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:06:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:06:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:06:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:06:18.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:18.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:18.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:06:18.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:06:18.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:06:18.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:18.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:18.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:06:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:06:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:18.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:18.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:18.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:06:18.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:18.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:06:18.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:06:18.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:18.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:18.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:06:18.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:06:18.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:18.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:18.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:06:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:06:18.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:18.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:18.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:18.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:06:19.020 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:06:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:06:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:06:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:06:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:06:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:06:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:06:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:06:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:06:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:06:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:06:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:19.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:06:19.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:19.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:19.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:19.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:06:19.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:19.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:06:19.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:06:19.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:06:19.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:06:19.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:06:19.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:06:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:06:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:06:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:06:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:06:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:06:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:06:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:06:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:06:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:06:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:19.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:06:19.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:06:19.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:06:19.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:06:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:06:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 42 2021-02-20T08:06:19.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:19.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:19.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:19.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:19.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:19.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:06:19.838 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:06:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=42 2021-02-20T08:06:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:06:20.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:20.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:20.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:20.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:20.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:20.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:20.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=38 2021-02-20T08:06:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 38 2021-02-20T08:06:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:06:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:20.677 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:20.681 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=38 2021-02-20T08:06:20.681 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 38 2021-02-20T08:06:20.681 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:06:21.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:21.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:21.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:21.566 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:21.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:06:21.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:06:21.572 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:06:21.572 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:06:21.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 2021-02-20T08:06:21.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:06:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:23.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:06:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:06:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:06:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:06:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 2021-02-20T08:06:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:06:25.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:25.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:25.294 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:06:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:06:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:06:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:06:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 2021-02-20T08:06:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:06:27.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:27.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:27.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=42 2021-02-20T08:06:27.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 42 = 42 2021-02-20T08:06:27.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:27.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2021-02-20T08:06:27.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:27.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:27.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:27.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10 2021-02-20T08:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:06:28.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2021-02-20T08:06:28.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 20 2021-02-20T08:06:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:06:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:06:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:06:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:06:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:06:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:06:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:06:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:05:29 2021 2021-02-20T08:06:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:06:28.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:06:28.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:06:28.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:06:28.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:06:28.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:06:28.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:28.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snap9-clean ] 2021-02-20T08:06:28.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T08:06:28.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:28.557 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:28.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9-clean 2021-02-20T08:06:28.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:28.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:28.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:28.559 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:28.559 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:29.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:30.670 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:30.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:30.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T08:06:30.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:30.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:30.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10 2021-02-20T08:06:30.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:30.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:30.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:30.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:30.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:30.879 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:06:30.872+0000 7ff4ec4a7700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T08:06:30.880 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:06:30.876+0000 7ff4e8ca0700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T08:06:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:33.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ clean_snap_name=snap10-clean 2021-02-20T08:06:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_snap cluster2 mirror test snap10-clean 2021-02-20T08:06:33.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:33.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:33.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:33.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10-clean 2021-02-20T08:06:33.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2021-02-20T08:06:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-02-20T08:06:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-02-20T08:06:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=started 2021-02-20T08:06:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:06:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:06:34.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_replay_state cluster1 mirror test started 2021-02-20T08:06:34.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:34.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=started 2021-02-20T08:06:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_result 2021-02-20T08:06:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=stopped 2021-02-20T08:06:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-02-20T08:06:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_instance=cluster1 2021-02-20T08:06:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ + shift 2021-02-20T08:06:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance=cluster1 2021-02-20T08:06:34.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ localgrep loop_instance 2021-02-20T08:06:34.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: -i state 2021-02-20T08:06:34.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:06:34.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ cluster1 != cluster1 ] 2021-02-20T08:06:34.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 0 3 2021-02-20T08:06:34.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-02-20T08:06:34.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:0 2021-02-20T08:06:34.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:34.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:34.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-02-20T08:06:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:06:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:06:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-02-20T08:06:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:0 2021-02-20T08:06:34.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:34.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:34.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-02-20T08:06:34.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:0 2021-02-20T08:06:34.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 0 = cluster1:0 ] 2021-02-20T08:06:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=0 2021-02-20T08:06:34.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=0 2021-02-20T08:06:34.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-02-20T08:06:34.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:34.943 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:34.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-02-20T08:06:34.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-02-20T08:06:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:06:35.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-02-20T08:06:35.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:1 2021-02-20T08:06:35.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-02-20T08:06:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:06:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:06:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-02-20T08:06:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:1 2021-02-20T08:06:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-02-20T08:06:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:1 2021-02-20T08:06:35.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:35.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:35.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:35.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 1 = cluster1:1 ] 2021-02-20T08:06:35.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=1 2021-02-20T08:06:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=1 2021-02-20T08:06:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-02-20T08:06:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-02-20T08:06:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-02-20T08:06:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-02-20T08:06:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd mirror status : get status for rbd mirror 2021-02-20T08:06:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:admin_socket: invalid command 2021-02-20T08:06:35.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-02-20T08:06:35.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1:2 2021-02-20T08:06:35.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:06:35.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:35.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-02-20T08:06:35.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:06:35.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=cluster 2021-02-20T08:06:35.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:35.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval cluster=cluster1 2021-02-20T08:06:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:06:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-02-20T08:06:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1:2 2021-02-20T08:06:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance 2021-02-20T08:06:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-02-20T08:06:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local val=cluster1:2 2021-02-20T08:06:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster_var_name=local_cluster 2021-02-20T08:06:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local instance_var_name=instance 2021-02-20T08:06:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cluster=cluster1 2021-02-20T08:06:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 2 = cluster1:2 ] 2021-02-20T08:06:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval local_cluster=cluster1 2021-02-20T08:06:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local_cluster=cluster1 2021-02-20T08:06:35.177 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ eval instance=2 2021-02-20T08:06:35.177 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ instance=2 2021-02-20T08:06:35.177 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-02-20T08:06:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-02-20T08:06:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-02-20T08:06:35.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:35.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ status_result= "state": "Replaying" 2021-02-20T08:06:35.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo "state": "Replaying" 2021-02-20T08:06:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ grep -i Replaying 2021-02-20T08:06:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=started 2021-02-20T08:06:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test started = started 2021-02-20T08:06:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:35.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-02-20T08:06:35.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:06:35.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:35.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:35.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:35.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = journal ] 2021-02-20T08:06:35.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ snapshot = snapshot ] 2021-02-20T08:06:35.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-02-20T08:06:35.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_cluster=cluster1 2021-02-20T08:06:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster2-mirror-test.status 2021-02-20T08:06:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster2-mirror-test.status 2021-02-20T08:06:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:06:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local status_log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mkfname cluster1-mirror-test.status 2021-02-20T08:06:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo cluster1-mirror-test.status 2021-02-20T08:06:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -e s|[/ ]|_|g 2021-02-20T08:06:35.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local local_status_log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:35.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-02-20T08:06:35.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:35.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-02-20T08:06:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: "state": "Replaying" 2021-02-20T08:06:35.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Snapshot ID: 45 2021-02-20T08:06:35.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:35.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:06:35.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:35.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:35.814 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.status 2021-02-20T08:06:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-02-20T08:06:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:35.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-02-20T08:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snapshot_id=45 2021-02-20T08:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:35.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.2 2021-02-20T08:06:36.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:36.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:36.440 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=42 2021-02-20T08:06:36.440 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 45 = 42 2021-02-20T08:06:36.440 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.4 2021-02-20T08:06:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:36.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:36.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:36.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:06:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:06:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:06:36.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:06:36.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 45 = 2021-02-20T08:06:36.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.8 2021-02-20T08:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:37.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:37.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:37.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:37.728 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:37.728 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:37.728 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:37.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ true 2021-02-20T08:06:37.813 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:37.817 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:-:1.1: Document is empty 2021-02-20T08:06:37.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: 2021-02-20T08:06:37.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:^ 2021-02-20T08:06:37.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id= 2021-02-20T08:06:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 45 = 2021-02-20T08:06:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1.6 2021-02-20T08:06:39.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:39.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:39.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:39.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:39.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local log=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.status 2021-02-20T08:06:39.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-02-20T08:06:39.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-02-20T08:06:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-02-20T08:06:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local primary_snapshot_id=45 2021-02-20T08:06:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test 45 = 45 2021-02-20T08:06:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2021-02-20T08:06:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:06:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10-clean 2021-02-20T08:06:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:06:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:06:40.506 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2021-02-20T08:06:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:rbd image 'test': 2021-02-20T08:06:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: size 512 MiB in 128 objects 2021-02-20T08:06:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: order 22 (4 MiB objects) 2021-02-20T08:06:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: snapshot_count: 21 2021-02-20T08:06:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: id: 109ec448d02 2021-02-20T08:06:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: block_name_prefix: rbd_data.109ec448d02 2021-02-20T08:06:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: format: 2 2021-02-20T08:06:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: features: layering, exclusive-lock, object-map, fast-diff, non-primary 2021-02-20T08:06:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: op_features: 2021-02-20T08:06:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: flags: 2021-02-20T08:06:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: create_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:06:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: access_timestamp: Sat Feb 20 08:06:30 2021 2021-02-20T08:06:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: modify_timestamp: Sat Feb 20 07:56:07 2021 2021-02-20T08:06:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: protected: False 2021-02-20T08:06:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring state: enabled 2021-02-20T08:06:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring mode: snapshot 2021-02-20T08:06:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring global id: eb5ee405-f45d-4109-b887-4acc3932b673 2021-02-20T08:06:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout: mirroring primary: false 2021-02-20T08:06:40.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:40.584 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 1 10 2021-02-20T08:06:40.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap1 2021-02-20T08:06:40.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap1 2021-02-20T08:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1 2021-02-20T08:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:41.915 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:06:41.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:06:41.917 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:06:41.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:06:41.911+0000 7fbbe3186700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 26353) UID: 1000 2021-02-20T08:06:42.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:06:42.687 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:42.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:42.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:43.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap1-clean 2021-02-20T08:06:43.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap1-clean 2021-02-20T08:06:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap1-clean 2021-02-20T08:06:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:43.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:43.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:43.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:43.826 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:06:43.819+0000 7f8029c96700 -1 bdev(0x55ac9e4aa000 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 1 2021-02-20T08:06:43.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:43.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:44.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:44.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:45.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap2 2021-02-20T08:06:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap2 2021-02-20T08:06:45.250 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:45.250 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:45.251 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2 2021-02-20T08:06:45.251 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:45.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:45.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:46.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:47.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:47.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap2-clean 2021-02-20T08:06:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap2-clean 2021-02-20T08:06:47.436 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:47.436 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:47.436 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap2-clean 2021-02-20T08:06:47.436 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:47.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:47.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:47.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:47.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:48.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:49.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:49.244 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:49.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:49.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap3 2021-02-20T08:06:49.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap3 2021-02-20T08:06:49.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:49.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:49.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3 2021-02-20T08:06:49.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:49.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:49.918 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:06:49.918 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:06:49.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:06:49.920 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:06:49.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:06:49.921 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:06:49.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:06:49.926 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T08:06:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:51.514 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:51.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:51.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:51.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:51.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap3-clean 2021-02-20T08:06:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap3-clean 2021-02-20T08:06:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap3-clean 2021-02-20T08:06:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:51.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:52.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:53.349 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:06:53.345+0000 7f1c6b5ce700 -1 bdev(0x555a508f6000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T08:06:53.350 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:06:53.345+0000 7f1c6bdcf700 -1 bdev(0x555a508f6000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2021-02-20T08:06:53.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:53.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap4 2021-02-20T08:06:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap4 2021-02-20T08:06:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4 2021-02-20T08:06:54.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:54.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:54.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:54.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:54.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:54.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:56.049 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:56.458 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap4-clean 2021-02-20T08:06:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap4-clean 2021-02-20T08:06:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:56.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:56.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap4-clean 2021-02-20T08:06:56.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:56.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:56.460 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:57.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:57.251 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:58.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:06:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap5 2021-02-20T08:06:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap5 2021-02-20T08:06:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:06:58.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:06:58.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5 2021-02-20T08:06:58.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:06:58.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:58.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:58.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:58.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:06:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:06:59.496 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:06:59.584 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:06:59.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:06:59.585 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:06:59.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:06:59.579+0000 7f08c0321700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 23588) UID: 1000 2021-02-20T08:06:59.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:07:00.437 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:00.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:00.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap5-clean 2021-02-20T08:07:00.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap5-clean 2021-02-20T08:07:00.847 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:00.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:00.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap5-clean 2021-02-20T08:07:00.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:00.848 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:01.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:01.722 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:02.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:02.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:03.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap6 2021-02-20T08:07:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap6 2021-02-20T08:07:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6 2021-02-20T08:07:03.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:03.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:03.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:04.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:05.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:05.344 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:05.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap6-clean 2021-02-20T08:07:05.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap6-clean 2021-02-20T08:07:05.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:05.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:05.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap6-clean 2021-02-20T08:07:05.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:05.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:05.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:05.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:05.347 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:05.496 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:07:05.487+0000 7ff4e9ca2700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2021-02-20T08:07:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:06.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:06.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:07:06.587 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:07:06.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:07:06.588 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:07:06.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:07:06.589 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:07:06.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:07:06.592 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T08:07:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:07.178 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap7 2021-02-20T08:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap7 2021-02-20T08:07:07.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:07.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7 2021-02-20T08:07:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:08.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:08.397 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:09.480 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:09.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:09.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap7-clean 2021-02-20T08:07:09.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap7-clean 2021-02-20T08:07:09.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:09.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:09.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap7-clean 2021-02-20T08:07:09.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:10.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:10.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:11.841 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:11.845 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:12.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:12.252 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:12.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap8 2021-02-20T08:07:12.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap8 2021-02-20T08:07:12.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:12.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:12.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8 2021-02-20T08:07:12.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:12.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:12.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:12.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:12.255 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:12.868 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T08:07:12.867+0000 7fc26239e700 -1 bdev(0x56116bcd8000 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2021-02-20T08:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:14.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:14.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap8-clean 2021-02-20T08:07:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap8-clean 2021-02-20T08:07:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap8-clean 2021-02-20T08:07:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:14.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:16.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:16.667 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap9 2021-02-20T08:07:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap9 2021-02-20T08:07:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9 2021-02-20T08:07:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:16.796 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:17.340 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:07:17.339+0000 7ff4ec4a7700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2021-02-20T08:07:17.341 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:07:17.339+0000 7ff4e9ca2700 -1 bdev(0x55dafd51a000 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2021-02-20T08:07:17.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:18.221 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T08:07:18.212+0000 7f1c6cdd1700 -1 bdev(0x555a508f6000 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2021-02-20T08:07:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap9-clean 2021-02-20T08:07:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap9-clean 2021-02-20T08:07:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap9-clean 2021-02-20T08:07:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:19.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:20.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap10 2021-02-20T08:07:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap10 2021-02-20T08:07:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10 2021-02-20T08:07:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:22.155 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:23.451 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap10-clean 2021-02-20T08:07:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ compare_image_snaps mirror test snap10-clean 2021-02-20T08:07:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap_name=snap10-clean 2021-02-20T08:07:23.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local ret=0 2021-02-20T08:07:23.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local rmt_export=/tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:23.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local loc_export=/tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export 2021-02-20T08:07:24.629 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:24.633 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:25.603 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-02-20T08:07:25.607 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ cmp /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rm -f /tmp/tmp.d0Pc51qzr5/cluster2-mirror-test.export /tmp/tmp.d0Pc51qzr5/cluster1-mirror-test.export 2021-02-20T08:07:26.015 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:26.017 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 1 10 2021-02-20T08:07:26.020 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap1 2021-02-20T08:07:26.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap1 2021-02-20T08:07:26.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:26.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap1 2021-02-20T08:07:26.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2021-02-20T08:07:26.765 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:26.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap1-clean 2021-02-20T08:07:26.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2021-02-20T08:07:26.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:26.791 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap1-clean 2021-02-20T08:07:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2021-02-20T08:07:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:27.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap2 2021-02-20T08:07:27.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap2 2021-02-20T08:07:27.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:27.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:27.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:27.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap2 2021-02-20T08:07:27.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2021-02-20T08:07:28.795 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:28.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap2-clean 2021-02-20T08:07:28.836 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2021-02-20T08:07:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap2-clean 2021-02-20T08:07:28.837 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2021-02-20T08:07:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap3 2021-02-20T08:07:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap3 2021-02-20T08:07:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap3 2021-02-20T08:07:29.900 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2021-02-20T08:07:30.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:30.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap3-clean 2021-02-20T08:07:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2021-02-20T08:07:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap3-clean 2021-02-20T08:07:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2021-02-20T08:07:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap4 2021-02-20T08:07:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap4 2021-02-20T08:07:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap4 2021-02-20T08:07:31.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2021-02-20T08:07:32.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap4-clean 2021-02-20T08:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2021-02-20T08:07:32.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:32.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:32.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:32.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap4-clean 2021-02-20T08:07:32.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2021-02-20T08:07:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap5 2021-02-20T08:07:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap5 2021-02-20T08:07:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap5 2021-02-20T08:07:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2021-02-20T08:07:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap5-clean 2021-02-20T08:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2021-02-20T08:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap5-clean 2021-02-20T08:07:34.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2021-02-20T08:07:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap6 2021-02-20T08:07:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap6 2021-02-20T08:07:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:35.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap6 2021-02-20T08:07:35.810 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2021-02-20T08:07:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap6-clean 2021-02-20T08:07:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2021-02-20T08:07:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap6-clean 2021-02-20T08:07:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2021-02-20T08:07:37.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:37.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap7 2021-02-20T08:07:37.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap7 2021-02-20T08:07:37.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:37.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:37.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:37.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap7 2021-02-20T08:07:37.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2021-02-20T08:07:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap7-clean 2021-02-20T08:07:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2021-02-20T08:07:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap7-clean 2021-02-20T08:07:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2021-02-20T08:07:39.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap8 2021-02-20T08:07:39.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap8 2021-02-20T08:07:39.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:39.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:39.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:39.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap8 2021-02-20T08:07:39.853 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2021-02-20T08:07:40.876 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:40.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap8-clean 2021-02-20T08:07:40.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2021-02-20T08:07:40.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:40.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:40.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:40.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap8-clean 2021-02-20T08:07:40.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2021-02-20T08:07:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap9 2021-02-20T08:07:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap9 2021-02-20T08:07:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap9 2021-02-20T08:07:41.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2021-02-20T08:07:42.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:43.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap9-clean 2021-02-20T08:07:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2021-02-20T08:07:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:43.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap9-clean 2021-02-20T08:07:43.044 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2021-02-20T08:07:43.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap10 2021-02-20T08:07:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap10 2021-02-20T08:07:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap10 2021-02-20T08:07:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2021-02-20T08:07:44.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:44.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap10-clean 2021-02-20T08:07:44.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2021-02-20T08:07:44.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:44.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:44.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:44.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local snap=snap10-clean 2021-02-20T08:07:44.990 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2021-02-20T08:07:45.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing snap: 100% complete...done. 2021-02-20T08:07:45.796 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:07:45.794+0000 7ff4f84bf700 -1 reset not still connected to 0x55db12bb0f70 2021-02-20T08:07:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_image_retry cluster2 mirror test 2021-02-20T08:07:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0 2021-02-20T08:07:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ remove_image cluster2 mirror test 2021-02-20T08:07:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:07:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2021-02-20T08:07:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 rm mirror/test 2021-02-20T08:07:47.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2021-02-20T08:07:47.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:07:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ wait_for_image_present cluster1 mirror test deleted 2021-02-20T08:07:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:47.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:47.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local state=deleted 2021-02-20T08:07:47.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id= 2021-02-20T08:07:47.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local s 2021-02-20T08:07:47.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 2021-02-20T08:07:47.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:07:47.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:47.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:47.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:47.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:07:47.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:07:48.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image_id=109ec448d02 2021-02-20T08:07:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 0.1 2021-02-20T08:07:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:07:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:07:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:07:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:07:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:07:48.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:07:48.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:48.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:48.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:48.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:07:48.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:07:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id=109ec448d02 2021-02-20T08:07:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 109ec448d02 2021-02-20T08:07:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -z 109ec448d02 -o 109ec448d02 = 109ec448d02 2021-02-20T08:07:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=present 2021-02-20T08:07:48.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = present 2021-02-20T08:07:48.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 1 2021-02-20T08:07:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:07:49.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:49.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:49.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:07:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:07:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:07:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:07:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:07:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:07:49.229 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:07:49.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id=109ec448d02 2021-02-20T08:07:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 109ec448d02 2021-02-20T08:07:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -z 109ec448d02 -o 109ec448d02 = 109ec448d02 2021-02-20T08:07:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=present 2021-02-20T08:07:49.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = present 2021-02-20T08:07:49.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 2 2021-02-20T08:07:51.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:07:51.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:51.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:07:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:07:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:07:51.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:07:51.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:07:51.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:51.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:51.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:51.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:07:51.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:07:51.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id=109ec448d02 2021-02-20T08:07:51.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 109ec448d02 2021-02-20T08:07:51.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -z 109ec448d02 -o 109ec448d02 = 109ec448d02 2021-02-20T08:07:51.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=present 2021-02-20T08:07:51.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = present 2021-02-20T08:07:51.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 4 2021-02-20T08:07:55.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:07:55.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:55.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:55.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:55.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:07:55.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:07:55.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:07:55.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:07:55.393 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:07:55.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:07:55.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:07:55.394 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:07:55.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:07:55.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:07:55.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id=109ec448d02 2021-02-20T08:07:55.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 109ec448d02 2021-02-20T08:07:55.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -z 109ec448d02 -o 109ec448d02 = 109ec448d02 2021-02-20T08:07:55.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=present 2021-02-20T08:07:55.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = present 2021-02-20T08:07:55.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T08:08:03.469 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:08:03.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:08:03.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:03.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:08:03.470 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:08:03.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:08:03.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:08:03.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:08:03.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:08:03.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:08:03.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:03.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:08:03.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:08:03.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:08:03.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id=109ec448d02 2021-02-20T08:08:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 109ec448d02 2021-02-20T08:08:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -z 109ec448d02 -o 109ec448d02 = 109ec448d02 2021-02-20T08:08:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_state=present 2021-02-20T08:08:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = present 2021-02-20T08:08:03.546 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sleep 8 2021-02-20T08:08:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test_image_present cluster1 mirror test deleted 109ec448d02 2021-02-20T08:08:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:08:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:08:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local test_state=deleted 2021-02-20T08:08:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image_id=109ec448d02 2021-02-20T08:08:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_state=deleted 2021-02-20T08:08:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local current_image_id 2021-02-20T08:08:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ get_image_id cluster1 mirror test 2021-02-20T08:08:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster1 2021-02-20T08:08:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=test 2021-02-20T08:08:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-02-20T08:08:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-02-20T08:08:11.609 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:rbd: error opening image test: (2) No such file or directory 2021-02-20T08:08:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ current_image_id= 2021-02-20T08:08:11.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test -n 2021-02-20T08:08:11.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ test deleted = deleted 2021-02-20T08:08:11.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:08:11.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ testlog TEST: create many images 2021-02-20T08:08:11.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ tee -a /tmp/tmp.d0Pc51qzr5/rbd-mirror.test.log 2021-02-20T08:08:11.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ date +%F %T 2021-02-20T08:08:11.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ echo 2021-02-20 08:08:11 TEST: create many images 2021-02-20T08:08:11.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:2021-02-20 08:08:11 TEST: create many images 2021-02-20T08:08:11.672 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ snap_name=snap 2021-02-20T08:08:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ seq 1 50 2021-02-20T08:08:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_1 2021-02-20T08:08:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 snapshot 128M 2021-02-20T08:08:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_1 2021-02-20T08:08:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:08:11.677 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.677 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_1 128M 2021-02-20T08:08:11.677 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:11.677 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_1 2021-02-20T08:08:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:08:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:08:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:08:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:11.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_1 2021-02-20T08:08:11.754 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:08:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_1 snapshot 2021-02-20T08:08:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_1 2021-02-20T08:08:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2021-02-20T08:08:11.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:08:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:08:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_1 2021-02-20T08:08:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_1 2021-02-20T08:08:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:12.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:08:12.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:08:12.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:12.477 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 33719296 bytes 2021-02-20T08:08:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 69735424 bytes 2021-02-20T08:08:13.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 84535296 bytes 2021-02-20T08:08:13.193 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 85129216 bytes 2021-02-20T08:08:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 125713408 bytes 2021-02-20T08:08:13.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 148163584 bytes 2021-02-20T08:08:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 231536640 bytes 2021-02-20T08:08:14.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 287683584 bytes 2021-02-20T08:08:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 332163072 bytes 2021-02-20T08:08:15.569 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 371084288 bytes 2021-02-20T08:08:16.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 435638272 bytes 2021-02-20T08:08:16.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 503849984 bytes 2021-02-20T08:08:16.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 507371520 bytes 2021-02-20T08:08:17.254 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 547910656 bytes 2021-02-20T08:08:17.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 594278400 bytes 2021-02-20T08:08:18.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 643734528 bytes 2021-02-20T08:08:18.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 707551232 bytes 2021-02-20T08:08:19.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 724704256 bytes 2021-02-20T08:08:19.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 788238336 bytes 2021-02-20T08:08:19.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 803899392 bytes 2021-02-20T08:08:20.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:08:20.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:08:20.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:08:20.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:08:20.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_2 2021-02-20T08:08:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 snapshot 128M 2021-02-20T08:08:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_2 2021-02-20T08:08:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:08:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_2 128M 2021-02-20T08:08:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_2 2021-02-20T08:08:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:08:20.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:08:20.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:08:20.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:20.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_2 2021-02-20T08:08:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:08:20.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_2 snapshot 2021-02-20T08:08:20.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:20.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:20.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_2 2021-02-20T08:08:20.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:20.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2021-02-20T08:08:20.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:08:20.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:08:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_2 2021-02-20T08:08:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:20.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_2 2021-02-20T08:08:20.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:20.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=10 2021-02-20T08:08:20.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:08:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 34446336 bytes 2021-02-20T08:08:22.003 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 103363584 bytes 2021-02-20T08:08:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 152134656 bytes 2021-02-20T08:08:22.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 170697728 bytes 2021-02-20T08:08:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 243824640 bytes 2021-02-20T08:08:23.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 285028352 bytes 2021-02-20T08:08:23.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 287217664 bytes 2021-02-20T08:08:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 315411456 bytes 2021-02-20T08:08:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 391945216 bytes 2021-02-20T08:08:25.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 463043584 bytes 2021-02-20T08:08:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 554647552 bytes 2021-02-20T08:08:26.346 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 604618752 bytes 2021-02-20T08:08:26.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 651915264 bytes 2021-02-20T08:08:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 672839680 bytes 2021-02-20T08:08:27.275 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 695146496 bytes 2021-02-20T08:08:27.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 792043520 bytes 2021-02-20T08:08:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 858636288 bytes 2021-02-20T08:08:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 897483776 bytes 2021-02-20T08:08:29.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 907011072 bytes 2021-02-20T08:08:29.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 925462528 bytes 2021-02-20T08:08:29.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 976282624 bytes 2021-02-20T08:08:30.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1017661440 bytes 2021-02-20T08:08:30.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 1020910592 bytes 2021-02-20T08:08:30.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1053833216 bytes 2021-02-20T08:08:30.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1065290752 bytes 2021-02-20T08:08:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:08:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:08:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:08:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:08:30.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_3 2021-02-20T08:08:30.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 snapshot 128M 2021-02-20T08:08:30.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:30.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_3 2021-02-20T08:08:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:08:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_3 128M 2021-02-20T08:08:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_3 2021-02-20T08:08:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:08:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:08:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:08:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_3 2021-02-20T08:08:31.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:08:31.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_3 snapshot 2021-02-20T08:08:31.305 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_3 2021-02-20T08:08:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2021-02-20T08:08:31.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:08:31.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:08:31.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_3 2021-02-20T08:08:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_3 2021-02-20T08:08:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=11 2021-02-20T08:08:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:08:32.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 86505472 bytes 2021-02-20T08:08:33.200 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 137162752 bytes 2021-02-20T08:08:33.592 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:08:33.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:08:33.594 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:08:33.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:08:33.589+0000 7f9df323b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 28219) UID: 1000 2021-02-20T08:08:33.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 188036096 bytes 2021-02-20T08:08:33.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:08:34.351 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 244113408 bytes 2021-02-20T08:08:34.594 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:08:34.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:08:34.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:08:34.596 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:08:34.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:08:34.596 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:08:34.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:08:34.599 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T08:08:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 301986816 bytes 2021-02-20T08:08:35.241 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 333863936 bytes 2021-02-20T08:08:35.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 350917632 bytes 2021-02-20T08:08:35.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 399621120 bytes 2021-02-20T08:08:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 468970496 bytes 2021-02-20T08:08:37.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 495522816 bytes 2021-02-20T08:08:37.517 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 536126464 bytes 2021-02-20T08:08:37.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 557127680 bytes 2021-02-20T08:08:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 643312640 bytes 2021-02-20T08:08:38.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 694206464 bytes 2021-02-20T08:08:38.927 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6551984954866756) > weight (0.25) 2021-02-20T08:08:38.928 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4738655484658949) > weight (0.25) 2021-02-20T08:08:38.928 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:08:38.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:08:38.929 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:08:38.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:08:38.925+0000 7fa8c0ea8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 18996) UID: 1000 2021-02-20T08:08:39.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 710541312 bytes 2021-02-20T08:08:39.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:08:39.472 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 739041280 bytes 2021-02-20T08:08:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 774276096 bytes 2021-02-20T08:08:40.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 814780416 bytes 2021-02-20T08:08:40.692 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 844527616 bytes 2021-02-20T08:08:41.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 886727680 bytes 2021-02-20T08:08:41.548 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 924382208 bytes 2021-02-20T08:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 940738560 bytes 2021-02-20T08:08:42.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 990241792 bytes 2021-02-20T08:08:42.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1009280000 bytes 2021-02-20T08:08:42.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1057648640 bytes 2021-02-20T08:08:43.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:08:43.006 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:08:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:08:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:08:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_4 2021-02-20T08:08:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 snapshot 128M 2021-02-20T08:08:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_4 2021-02-20T08:08:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:08:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_4 128M 2021-02-20T08:08:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_4 2021-02-20T08:08:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:08:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:08:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:08:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_4 2021-02-20T08:08:43.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:08:43.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_4 snapshot 2021-02-20T08:08:43.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:43.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:43.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_4 2021-02-20T08:08:43.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:43.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2021-02-20T08:08:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:08:43.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:08:43.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_4 2021-02-20T08:08:43.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:43.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:43.544 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_4 2021-02-20T08:08:43.545 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:43.546 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:08:43.546 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:08:43.547 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 33873920 bytes 2021-02-20T08:08:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 93843456 bytes 2021-02-20T08:08:44.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 144913408 bytes 2021-02-20T08:08:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 145594368 bytes 2021-02-20T08:08:45.197 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 146237440 bytes 2021-02-20T08:08:45.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 163657728 bytes 2021-02-20T08:08:45.762 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 205229056 bytes 2021-02-20T08:08:46.164 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 273421312 bytes 2021-02-20T08:08:46.446 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 274033664 bytes 2021-02-20T08:08:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 312097792 bytes 2021-02-20T08:08:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 372707328 bytes 2021-02-20T08:08:47.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 379762688 bytes 2021-02-20T08:08:47.929 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:08:47.930 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:08:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:08:47.931 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:08:47.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:08:47.931 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:08:47.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:08:47.934 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T08:08:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 425359360 bytes 2021-02-20T08:08:48.301 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 448443392 bytes 2021-02-20T08:08:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 473385984 bytes 2021-02-20T08:08:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 504158208 bytes 2021-02-20T08:08:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 557365248 bytes 2021-02-20T08:08:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 621953024 bytes 2021-02-20T08:08:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 720931840 bytes 2021-02-20T08:08:50.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 737122304 bytes 2021-02-20T08:08:51.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 786497536 bytes 2021-02-20T08:08:51.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 819984384 bytes 2021-02-20T08:08:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 846468096 bytes 2021-02-20T08:08:51.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 864009216 bytes 2021-02-20T08:08:52.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 889255936 bytes 2021-02-20T08:08:52.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 889873408 bytes 2021-02-20T08:08:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2700 IOs, 950153216 bytes 2021-02-20T08:08:53.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2800 IOs, 957504512 bytes 2021-02-20T08:08:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2900 IOs, 1037361152 bytes 2021-02-20T08:08:54.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3000 IOs, 1121963008 bytes 2021-02-20T08:08:54.315 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3100 IOs, 1135293440 bytes 2021-02-20T08:08:54.642 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3200 IOs, 1174383616 bytes 2021-02-20T08:08:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3300 IOs, 1198148608 bytes 2021-02-20T08:08:55.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3400 IOs, 1271714816 bytes 2021-02-20T08:08:55.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:08:55.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:08:55.550 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:08:55.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:08:55.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_5 2021-02-20T08:08:55.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 snapshot 128M 2021-02-20T08:08:55.551 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_5 2021-02-20T08:08:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:08:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_5 128M 2021-02-20T08:08:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:55.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:55.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_5 2021-02-20T08:08:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:08:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:08:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:08:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:08:55.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_5 2021-02-20T08:08:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:08:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_5 snapshot 2021-02-20T08:08:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_5 2021-02-20T08:08:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:08:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2021-02-20T08:08:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:08:56.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:08:56.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_5 2021-02-20T08:08:56.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:08:56.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:08:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_5 2021-02-20T08:08:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:08:56.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=14 2021-02-20T08:08:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:08:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:08:57.599 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 72545280 bytes 2021-02-20T08:08:57.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 95283200 bytes 2021-02-20T08:08:58.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 138850304 bytes 2021-02-20T08:08:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 207281152 bytes 2021-02-20T08:08:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 263285760 bytes 2021-02-20T08:08:59.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 291830784 bytes 2021-02-20T08:08:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 350916608 bytes 2021-02-20T08:09:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 418048000 bytes 2021-02-20T08:09:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 467206144 bytes 2021-02-20T08:09:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 478054400 bytes 2021-02-20T08:09:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 518194176 bytes 2021-02-20T08:09:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 554432512 bytes 2021-02-20T08:09:02.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 580774912 bytes 2021-02-20T08:09:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 602982400 bytes 2021-02-20T08:09:02.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 635394048 bytes 2021-02-20T08:09:03.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 670546944 bytes 2021-02-20T08:09:03.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 692268032 bytes 2021-02-20T08:09:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 699347968 bytes 2021-02-20T08:09:03.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 701753344 bytes 2021-02-20T08:09:04.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 754048000 bytes 2021-02-20T08:09:04.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 775559168 bytes 2021-02-20T08:09:04.684 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 795167744 bytes 2021-02-20T08:09:05.046 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 837363712 bytes 2021-02-20T08:09:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 917679104 bytes 2021-02-20T08:09:05.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 964372480 bytes 2021-02-20T08:09:06.261 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 976417792 bytes 2021-02-20T08:09:06.516 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2700 IOs, 1004421120 bytes 2021-02-20T08:09:07.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2800 IOs, 1075553280 bytes 2021-02-20T08:09:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2900 IOs, 1089610752 bytes 2021-02-20T08:09:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3000 IOs, 1194783744 bytes 2021-02-20T08:09:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3100 IOs, 1247216640 bytes 2021-02-20T08:09:08.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3200 IOs, 1310802944 bytes 2021-02-20T08:09:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3300 IOs, 1313967104 bytes 2021-02-20T08:09:09.488 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3400 IOs, 1335657472 bytes 2021-02-20T08:09:10.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3500 IOs, 1409580032 bytes 2021-02-20T08:09:10.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3600 IOs, 1441660928 bytes 2021-02-20T08:09:10.347 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T08:09:10.340+0000 7ff4f84bf700 -1 reset not still connected to 0x55db12bb23c0 2021-02-20T08:09:10.818 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3700 IOs, 1509206016 bytes 2021-02-20T08:09:10.877 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:10.877 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:10.878 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:10.878 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:10.878 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_6 2021-02-20T08:09:10.878 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 snapshot 128M 2021-02-20T08:09:10.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:10.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:10.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_6 2021-02-20T08:09:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_6 128M 2021-02-20T08:09:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:10.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_6 2021-02-20T08:09:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_6 2021-02-20T08:09:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:11.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_6 snapshot 2021-02-20T08:09:11.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:11.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:11.248 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_6 2021-02-20T08:09:11.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:11.249 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2021-02-20T08:09:11.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_6 2021-02-20T08:09:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_6 2021-02-20T08:09:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:09:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:12.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 607232 bytes 2021-02-20T08:09:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 28059648 bytes 2021-02-20T08:09:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 67860480 bytes 2021-02-20T08:09:13.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 140809216 bytes 2021-02-20T08:09:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 172198912 bytes 2021-02-20T08:09:14.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 265054208 bytes 2021-02-20T08:09:14.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 295881728 bytes 2021-02-20T08:09:15.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 349335552 bytes 2021-02-20T08:09:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 363535360 bytes 2021-02-20T08:09:15.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 411429888 bytes 2021-02-20T08:09:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 449573888 bytes 2021-02-20T08:09:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 462504960 bytes 2021-02-20T08:09:16.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:16.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:16.867 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_7 2021-02-20T08:09:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 snapshot 128M 2021-02-20T08:09:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_7 2021-02-20T08:09:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:16.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:16.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_7 128M 2021-02-20T08:09:16.870 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:16.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.871 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:16.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_7 2021-02-20T08:09:16.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:16.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:16.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:16.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:16.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_7 2021-02-20T08:09:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_7 snapshot 2021-02-20T08:09:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_7 2021-02-20T08:09:17.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:17.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2021-02-20T08:09:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_7 2021-02-20T08:09:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:17.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:17.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_7 2021-02-20T08:09:17.874 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:17.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=7 2021-02-20T08:09:17.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:17.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:18.631 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 27389952 bytes 2021-02-20T08:09:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 75317248 bytes 2021-02-20T08:09:19.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 129776640 bytes 2021-02-20T08:09:19.668 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 146304000 bytes 2021-02-20T08:09:19.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 148590592 bytes 2021-02-20T08:09:20.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 203328512 bytes 2021-02-20T08:09:20.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 278963200 bytes 2021-02-20T08:09:21.199 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 309614592 bytes 2021-02-20T08:09:21.687 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 374080512 bytes 2021-02-20T08:09:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 427357184 bytes 2021-02-20T08:09:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 453817344 bytes 2021-02-20T08:09:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 475773952 bytes 2021-02-20T08:09:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 518283264 bytes 2021-02-20T08:09:23.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 538984448 bytes 2021-02-20T08:09:23.846 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 579321856 bytes 2021-02-20T08:09:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 601896960 bytes 2021-02-20T08:09:24.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 602504192 bytes 2021-02-20T08:09:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 614298624 bytes 2021-02-20T08:09:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 614924288 bytes 2021-02-20T08:09:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:24.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:24.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:24.880 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:24.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_8 2021-02-20T08:09:24.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 snapshot 128M 2021-02-20T08:09:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_8 2021-02-20T08:09:24.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.884 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:24.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:24.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_8 128M 2021-02-20T08:09:24.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.888 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_8 2021-02-20T08:09:24.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:24.889 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:24.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:24.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:24.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_8 2021-02-20T08:09:24.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_8 snapshot 2021-02-20T08:09:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_8 2021-02-20T08:09:24.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:24.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2021-02-20T08:09:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:25.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_8 2021-02-20T08:09:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_8 2021-02-20T08:09:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=6 2021-02-20T08:09:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:25.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 69048320 bytes 2021-02-20T08:09:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 180571136 bytes 2021-02-20T08:09:27.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 209195008 bytes 2021-02-20T08:09:27.447 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 265125888 bytes 2021-02-20T08:09:27.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 329161728 bytes 2021-02-20T08:09:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 349991936 bytes 2021-02-20T08:09:28.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 424854528 bytes 2021-02-20T08:09:28.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 428560384 bytes 2021-02-20T08:09:29.188 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 444009472 bytes 2021-02-20T08:09:29.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 460260352 bytes 2021-02-20T08:09:29.650 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 461933568 bytes 2021-02-20T08:09:30.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 516198400 bytes 2021-02-20T08:09:30.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 536056832 bytes 2021-02-20T08:09:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 554926080 bytes 2021-02-20T08:09:31.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 623664128 bytes 2021-02-20T08:09:31.294 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:31.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:31.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:31.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:31.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_9 2021-02-20T08:09:31.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 snapshot 128M 2021-02-20T08:09:31.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:31.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_9 2021-02-20T08:09:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_9 128M 2021-02-20T08:09:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_9 2021-02-20T08:09:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:31.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:31.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:31.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:31.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_9 2021-02-20T08:09:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_9 snapshot 2021-02-20T08:09:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_9 2021-02-20T08:09:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2021-02-20T08:09:33.043 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:33.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_9 2021-02-20T08:09:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_9 2021-02-20T08:09:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:09:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 37037056 bytes 2021-02-20T08:09:33.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 46498816 bytes 2021-02-20T08:09:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 59042816 bytes 2021-02-20T08:09:34.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 68177920 bytes 2021-02-20T08:09:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 160269312 bytes 2021-02-20T08:09:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 198022144 bytes 2021-02-20T08:09:35.665 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 210945024 bytes 2021-02-20T08:09:36.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 283072512 bytes 2021-02-20T08:09:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 325548032 bytes 2021-02-20T08:09:36.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 331524096 bytes 2021-02-20T08:09:37.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 404540416 bytes 2021-02-20T08:09:38.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 466185216 bytes 2021-02-20T08:09:38.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:38.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_10 2021-02-20T08:09:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 snapshot 128M 2021-02-20T08:09:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_10 2021-02-20T08:09:38.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:38.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:38.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_10 128M 2021-02-20T08:09:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:38.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_10 2021-02-20T08:09:38.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:38.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:38.087 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:38.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:38.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_10 2021-02-20T08:09:38.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:38.236 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_10 snapshot 2021-02-20T08:09:38.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:38.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:38.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_10 2021-02-20T08:09:38.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:38.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2021-02-20T08:09:39.056 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_10 2021-02-20T08:09:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:39.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:39.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_10 2021-02-20T08:09:39.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:39.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:09:39.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:39.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 61424640 bytes 2021-02-20T08:09:40.190 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 106634240 bytes 2021-02-20T08:09:40.527 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 145789952 bytes 2021-02-20T08:09:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 219796480 bytes 2021-02-20T08:09:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 254411776 bytes 2021-02-20T08:09:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 309760000 bytes 2021-02-20T08:09:42.511 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 352882688 bytes 2021-02-20T08:09:42.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 400114688 bytes 2021-02-20T08:09:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 456430592 bytes 2021-02-20T08:09:43.981 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 538360832 bytes 2021-02-20T08:09:44.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_11 2021-02-20T08:09:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 snapshot 128M 2021-02-20T08:09:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_11 2021-02-20T08:09:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_11 128M 2021-02-20T08:09:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:44.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_11 2021-02-20T08:09:44.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:44.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:44.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_11 2021-02-20T08:09:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:44.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_11 snapshot 2021-02-20T08:09:44.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:44.181 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:44.182 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_11 2021-02-20T08:09:44.182 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:44.182 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_11 snapshot 2021-02-20T08:09:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:45.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:45.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_11 2021-02-20T08:09:45.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_11 2021-02-20T08:09:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:45.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:09:45.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:45.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 41754624 bytes 2021-02-20T08:09:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 82745344 bytes 2021-02-20T08:09:47.206 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 131174400 bytes 2021-02-20T08:09:47.729 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 184111104 bytes 2021-02-20T08:09:48.231 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 244135936 bytes 2021-02-20T08:09:48.559 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 269488128 bytes 2021-02-20T08:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 276893696 bytes 2021-02-20T08:09:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 360708096 bytes 2021-02-20T08:09:49.869 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 410459136 bytes 2021-02-20T08:09:50.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 428278784 bytes 2021-02-20T08:09:50.514 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 458539008 bytes 2021-02-20T08:09:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 506769408 bytes 2021-02-20T08:09:51.292 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 532185088 bytes 2021-02-20T08:09:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 545036288 bytes 2021-02-20T08:09:52.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 612931584 bytes 2021-02-20T08:09:52.872 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 682644480 bytes 2021-02-20T08:09:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 695330816 bytes 2021-02-20T08:09:53.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 762256384 bytes 2021-02-20T08:09:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 798654464 bytes 2021-02-20T08:09:54.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 815997952 bytes 2021-02-20T08:09:55.395 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 962952192 bytes 2021-02-20T08:09:55.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1042906112 bytes 2021-02-20T08:09:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 1056337920 bytes 2021-02-20T08:09:56.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1127251968 bytes 2021-02-20T08:09:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 1151480832 bytes 2021-02-20T08:09:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:09:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:09:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:09:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:09:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_12 2021-02-20T08:09:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 snapshot 128M 2021-02-20T08:09:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:57.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_12 2021-02-20T08:09:57.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:57.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:09:57.781 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_12 128M 2021-02-20T08:09:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_12 2021-02-20T08:09:57.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:09:57.784 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:09:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:09:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:09:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_12 2021-02-20T08:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_12 snapshot 2021-02-20T08:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_12 2021-02-20T08:09:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:09:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_12 snapshot 2021-02-20T08:09:59.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:09:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:09:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_12 2021-02-20T08:09:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:09:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:09:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_12 2021-02-20T08:09:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:09:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:09:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:09:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:09:59.623 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 11593728 bytes 2021-02-20T08:09:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 39236608 bytes 2021-02-20T08:10:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 76544000 bytes 2021-02-20T08:10:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 115944448 bytes 2021-02-20T08:10:01.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 179178496 bytes 2021-02-20T08:10:01.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 226883584 bytes 2021-02-20T08:10:02.018 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 231913472 bytes 2021-02-20T08:10:02.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 278412288 bytes 2021-02-20T08:10:02.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 314636288 bytes 2021-02-20T08:10:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 399003648 bytes 2021-02-20T08:10:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 458003456 bytes 2021-02-20T08:10:04.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:04.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_13 2021-02-20T08:10:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 snapshot 128M 2021-02-20T08:10:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_13 2021-02-20T08:10:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:04.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:04.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_13 128M 2021-02-20T08:10:04.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:04.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:04.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_13 2021-02-20T08:10:04.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_13 2021-02-20T08:10:04.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:04.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_13 snapshot 2021-02-20T08:10:04.342 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_13 2021-02-20T08:10:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_13 snapshot 2021-02-20T08:10:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_13 2021-02-20T08:10:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_13 2021-02-20T08:10:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:05.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:10:05.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:05.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 42945536 bytes 2021-02-20T08:10:06.250 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 137533440 bytes 2021-02-20T08:10:06.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 182051840 bytes 2021-02-20T08:10:07.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 218378240 bytes 2021-02-20T08:10:07.553 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 260057088 bytes 2021-02-20T08:10:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 274370560 bytes 2021-02-20T08:10:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 325740544 bytes 2021-02-20T08:10:08.704 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 379563008 bytes 2021-02-20T08:10:09.392 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 488185856 bytes 2021-02-20T08:10:10.005 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 543721472 bytes 2021-02-20T08:10:10.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 570002432 bytes 2021-02-20T08:10:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 614351872 bytes 2021-02-20T08:10:11.211 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 658524160 bytes 2021-02-20T08:10:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 782230528 bytes 2021-02-20T08:10:12.449 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 807734272 bytes 2021-02-20T08:10:12.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 870948864 bytes 2021-02-20T08:10:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_14 2021-02-20T08:10:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 snapshot 128M 2021-02-20T08:10:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_14 2021-02-20T08:10:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_14 128M 2021-02-20T08:10:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_14 2021-02-20T08:10:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:13.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:13.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_14 2021-02-20T08:10:13.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:13.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_14 snapshot 2021-02-20T08:10:13.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:13.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:13.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_14 2021-02-20T08:10:13.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_14 snapshot 2021-02-20T08:10:13.562 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_14 2021-02-20T08:10:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:13.605 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:13.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_14 2021-02-20T08:10:13.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:13.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=11 2021-02-20T08:10:13.607 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:13.607 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 30282752 bytes 2021-02-20T08:10:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 130287616 bytes 2021-02-20T08:10:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 136145920 bytes 2021-02-20T08:10:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 199484416 bytes 2021-02-20T08:10:15.600 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3213939643161353) > weight (0.25) 2021-02-20T08:10:15.601 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8922340198651179) > weight (0.25) 2021-02-20T08:10:15.601 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:10:15.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:10:15.602 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T08:10:15.676 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T08:10:15.671+0000 7f6be1e7e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 19094) UID: 1000 2021-02-20T08:10:15.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 242710528 bytes 2021-02-20T08:10:16.373 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 273413120 bytes 2021-02-20T08:10:16.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:10:16.630 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 276802560 bytes 2021-02-20T08:10:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 365064192 bytes 2021-02-20T08:10:17.899 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 423142400 bytes 2021-02-20T08:10:18.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 447513600 bytes 2021-02-20T08:10:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 485112832 bytes 2021-02-20T08:10:18.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5236805369722152) > weight (0.25) 2021-02-20T08:10:18.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2517800359933505) > weight (0.25) 2021-02-20T08:10:18.935 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:10:18.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:10:18.936 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:10:18.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:10:18.931+0000 7f0738750700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 29737) UID: 1000 2021-02-20T08:10:18.991 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 506547200 bytes 2021-02-20T08:10:19.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:10:19.401 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 545366016 bytes 2021-02-20T08:10:19.603 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:10:19.604 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:10:19.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:10:19.604 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:10:19.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:10:19.605 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:10:19.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:10:19.628 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T08:10:19.691 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 557350912 bytes 2021-02-20T08:10:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 573671424 bytes 2021-02-20T08:10:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 612652032 bytes 2021-02-20T08:10:20.937 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:10:20.937 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:10:20.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:10:20.938 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:10:20.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:10:20.939 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:10:20.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:10:20.943 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T08:10:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 701625344 bytes 2021-02-20T08:10:21.455 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 722257920 bytes 2021-02-20T08:10:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 772826112 bytes 2021-02-20T08:10:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 851408896 bytes 2021-02-20T08:10:22.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 880724992 bytes 2021-02-20T08:10:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 966009856 bytes 2021-02-20T08:10:23.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 980728832 bytes 2021-02-20T08:10:24.585 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 1061301248 bytes 2021-02-20T08:10:24.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:24.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:24.613 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_15 2021-02-20T08:10:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 snapshot 128M 2021-02-20T08:10:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_15 2021-02-20T08:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_15 128M 2021-02-20T08:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_15 2021-02-20T08:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_15 2021-02-20T08:10:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_15 snapshot 2021-02-20T08:10:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_15 2021-02-20T08:10:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:24.702 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_15 snapshot 2021-02-20T08:10:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:25.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:25.310 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_15 2021-02-20T08:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_15 2021-02-20T08:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T08:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:26.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 3230720 bytes 2021-02-20T08:10:26.441 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 46846976 bytes 2021-02-20T08:10:26.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 109789184 bytes 2021-02-20T08:10:27.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 114923520 bytes 2021-02-20T08:10:27.634 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 153649152 bytes 2021-02-20T08:10:28.475 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 288734208 bytes 2021-02-20T08:10:28.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 337915904 bytes 2021-02-20T08:10:29.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 351035392 bytes 2021-02-20T08:10:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 359720960 bytes 2021-02-20T08:10:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 411912192 bytes 2021-02-20T08:10:30.560 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 470486016 bytes 2021-02-20T08:10:30.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 511880192 bytes 2021-02-20T08:10:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 592425984 bytes 2021-02-20T08:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 620683264 bytes 2021-02-20T08:10:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 621295616 bytes 2021-02-20T08:10:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 695993344 bytes 2021-02-20T08:10:33.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 736865280 bytes 2021-02-20T08:10:33.582 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 761267200 bytes 2021-02-20T08:10:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 804569088 bytes 2021-02-20T08:10:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_16 2021-02-20T08:10:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 snapshot 128M 2021-02-20T08:10:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_16 2021-02-20T08:10:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_16 128M 2021-02-20T08:10:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_16 2021-02-20T08:10:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:34.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_16 2021-02-20T08:10:34.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:34.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_16 snapshot 2021-02-20T08:10:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_16 2021-02-20T08:10:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:34.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_16 snapshot 2021-02-20T08:10:35.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_16 2021-02-20T08:10:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:35.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_16 2021-02-20T08:10:35.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=13 2021-02-20T08:10:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:35.304 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:35.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 24597504 bytes 2021-02-20T08:10:36.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 78627840 bytes 2021-02-20T08:10:36.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 92487680 bytes 2021-02-20T08:10:37.487 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 195508224 bytes 2021-02-20T08:10:37.958 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 249194496 bytes 2021-02-20T08:10:38.228 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 263595008 bytes 2021-02-20T08:10:38.715 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 317237248 bytes 2021-02-20T08:10:39.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 368336896 bytes 2021-02-20T08:10:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 433907712 bytes 2021-02-20T08:10:40.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 451041280 bytes 2021-02-20T08:10:40.558 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 503339008 bytes 2021-02-20T08:10:40.830 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 525291520 bytes 2021-02-20T08:10:41.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 564301824 bytes 2021-02-20T08:10:41.748 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 585695232 bytes 2021-02-20T08:10:42.357 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 659386368 bytes 2021-02-20T08:10:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 670872576 bytes 2021-02-20T08:10:42.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 689267712 bytes 2021-02-20T08:10:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 738733056 bytes 2021-02-20T08:10:43.901 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 787779584 bytes 2021-02-20T08:10:44.175 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 794070016 bytes 2021-02-20T08:10:44.661 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 849284096 bytes 2021-02-20T08:10:44.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 871446528 bytes 2021-02-20T08:10:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 898036736 bytes 2021-02-20T08:10:45.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2400 IOs, 926405632 bytes 2021-02-20T08:10:45.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2500 IOs, 927000576 bytes 2021-02-20T08:10:46.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2600 IOs, 952095744 bytes 2021-02-20T08:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2700 IOs, 961021952 bytes 2021-02-20T08:10:46.852 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2800 IOs, 979554304 bytes 2021-02-20T08:10:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2900 IOs, 980167680 bytes 2021-02-20T08:10:47.657 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3000 IOs, 1006294016 bytes 2021-02-20T08:10:48.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:3100 IOs, 1051616256 bytes 2021-02-20T08:10:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_17 2021-02-20T08:10:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 snapshot 128M 2021-02-20T08:10:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_17 2021-02-20T08:10:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_17 128M 2021-02-20T08:10:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_17 2021-02-20T08:10:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_17 2021-02-20T08:10:48.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:48.524 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_17 snapshot 2021-02-20T08:10:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_17 2021-02-20T08:10:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:48.525 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_17 snapshot 2021-02-20T08:10:49.074 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_17 2021-02-20T08:10:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:49.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_17 2021-02-20T08:10:49.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:49.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T08:10:49.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:49.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:49.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 26426368 bytes 2021-02-20T08:10:50.051 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 54500352 bytes 2021-02-20T08:10:50.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 63030272 bytes 2021-02-20T08:10:50.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 140524544 bytes 2021-02-20T08:10:51.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 164023296 bytes 2021-02-20T08:10:51.450 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 171954176 bytes 2021-02-20T08:10:51.856 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 209098752 bytes 2021-02-20T08:10:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 248666112 bytes 2021-02-20T08:10:52.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 333351936 bytes 2021-02-20T08:10:53.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 358389760 bytes 2021-02-20T08:10:53.634 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 388642816 bytes 2021-02-20T08:10:54.041 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 428136448 bytes 2021-02-20T08:10:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 473475072 bytes 2021-02-20T08:10:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 503159808 bytes 2021-02-20T08:10:55.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 551239680 bytes 2021-02-20T08:10:56.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 580952064 bytes 2021-02-20T08:10:56.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 627443712 bytes 2021-02-20T08:10:57.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 665290752 bytes 2021-02-20T08:10:58.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 680670208 bytes 2021-02-20T08:10:58.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:10:58.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:10:58.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:10:58.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:10:58.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_18 2021-02-20T08:10:58.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 snapshot 128M 2021-02-20T08:10:58.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:58.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_18 2021-02-20T08:10:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:10:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_18 128M 2021-02-20T08:10:58.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:58.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.124 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:58.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_18 2021-02-20T08:10:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:10:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:10:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:10:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:10:58.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_18 2021-02-20T08:10:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:10:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_18 snapshot 2021-02-20T08:10:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_18 2021-02-20T08:10:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:10:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_18 snapshot 2021-02-20T08:10:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:10:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:10:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_18 2021-02-20T08:10:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:10:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:10:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_18 2021-02-20T08:10:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:10:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:10:59.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:10:59.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:10:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 3018752 bytes 2021-02-20T08:11:00.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 22194176 bytes 2021-02-20T08:11:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 82553856 bytes 2021-02-20T08:11:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 146905088 bytes 2021-02-20T08:11:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 208992256 bytes 2021-02-20T08:11:02.757 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 264437760 bytes 2021-02-20T08:11:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 317458432 bytes 2021-02-20T08:11:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 376805376 bytes 2021-02-20T08:11:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 397133824 bytes 2021-02-20T08:11:04.834 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 420618240 bytes 2021-02-20T08:11:05.580 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 468413440 bytes 2021-02-20T08:11:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 500937728 bytes 2021-02-20T08:11:07.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:07.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:07.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:07.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:07.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_19 2021-02-20T08:11:07.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 snapshot 128M 2021-02-20T08:11:07.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:07.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_19 2021-02-20T08:11:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:07.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:07.115 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_19 128M 2021-02-20T08:11:07.116 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_19 2021-02-20T08:11:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_19 2021-02-20T08:11:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:11:07.202 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_19 snapshot 2021-02-20T08:11:07.202 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:07.202 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:07.202 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_19 2021-02-20T08:11:07.203 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:07.203 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_19 snapshot 2021-02-20T08:11:07.866 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:11:07.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:11:07.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_19 2021-02-20T08:11:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_19 2021-02-20T08:11:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:07.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:11:07.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:11:07.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 17832960 bytes 2021-02-20T08:11:09.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 99512320 bytes 2021-02-20T08:11:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 173511680 bytes 2021-02-20T08:11:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 196398080 bytes 2021-02-20T08:11:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 227503104 bytes 2021-02-20T08:11:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 257320960 bytes 2021-02-20T08:11:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 263531520 bytes 2021-02-20T08:11:11.789 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 300269568 bytes 2021-02-20T08:11:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 326091776 bytes 2021-02-20T08:11:12.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:12.903 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:12.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:12.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:12.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_20 2021-02-20T08:11:12.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 snapshot 128M 2021-02-20T08:11:12.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:12.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:12.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.905 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_20 2021-02-20T08:11:12.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:12.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:12.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_20 128M 2021-02-20T08:11:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_20 2021-02-20T08:11:12.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:12.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:12.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:12.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:12.909 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_20 2021-02-20T08:11:12.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:11:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_20 snapshot 2021-02-20T08:11:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_20 2021-02-20T08:11:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_20 snapshot 2021-02-20T08:11:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:11:13.270 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:11:13.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_20 2021-02-20T08:11:13.271 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:13.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:13.272 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_20 2021-02-20T08:11:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:11:13.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:11:13.274 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 66260992 bytes 2021-02-20T08:11:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 130720768 bytes 2021-02-20T08:11:14.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 141168640 bytes 2021-02-20T08:11:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 173897728 bytes 2021-02-20T08:11:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 200620032 bytes 2021-02-20T08:11:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 245978112 bytes 2021-02-20T08:11:17.398 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 304814080 bytes 2021-02-20T08:11:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 371105792 bytes 2021-02-20T08:11:18.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 419142656 bytes 2021-02-20T08:11:19.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 446192640 bytes 2021-02-20T08:11:19.750 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 465678336 bytes 2021-02-20T08:11:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 499830784 bytes 2021-02-20T08:11:20.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 530837504 bytes 2021-02-20T08:11:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 606483456 bytes 2021-02-20T08:11:22.355 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 640662528 bytes 2021-02-20T08:11:23.221 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 710506496 bytes 2021-02-20T08:11:23.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 762172416 bytes 2021-02-20T08:11:24.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 803462144 bytes 2021-02-20T08:11:25.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 844116992 bytes 2021-02-20T08:11:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:25.280 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:25.280 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:25.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_21 2021-02-20T08:11:25.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 snapshot 128M 2021-02-20T08:11:25.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:25.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_21 2021-02-20T08:11:25.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:25.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:25.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_21 128M 2021-02-20T08:11:25.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:25.286 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:25.287 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_21 2021-02-20T08:11:25.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_21 2021-02-20T08:11:25.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:11:25.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_21 snapshot 2021-02-20T08:11:25.365 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_21 2021-02-20T08:11:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_21 snapshot 2021-02-20T08:11:25.881 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:11:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:11:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_21 2021-02-20T08:11:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_21 2021-02-20T08:11:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=14 2021-02-20T08:11:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:11:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 19988480 bytes 2021-02-20T08:11:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 121553920 bytes 2021-02-20T08:11:28.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 213786624 bytes 2021-02-20T08:11:28.492 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 242095104 bytes 2021-02-20T08:11:29.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 324476928 bytes 2021-02-20T08:11:29.668 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 325668864 bytes 2021-02-20T08:11:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 352161792 bytes 2021-02-20T08:11:30.636 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 367396864 bytes 2021-02-20T08:11:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 418929664 bytes 2021-02-20T08:11:31.843 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 453886976 bytes 2021-02-20T08:11:32.456 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 489263104 bytes 2021-02-20T08:11:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 573675520 bytes 2021-02-20T08:11:33.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 616740864 bytes 2021-02-20T08:11:34.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 683460608 bytes 2021-02-20T08:11:35.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 738663424 bytes 2021-02-20T08:11:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 753694720 bytes 2021-02-20T08:11:36.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 776834048 bytes 2021-02-20T08:11:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 824015872 bytes 2021-02-20T08:11:37.710 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 852486144 bytes 2021-02-20T08:11:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 896168960 bytes 2021-02-20T08:11:39.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 946502656 bytes 2021-02-20T08:11:39.654 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 989179904 bytes 2021-02-20T08:11:39.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:39.928 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:39.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_22 2021-02-20T08:11:39.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 snapshot 128M 2021-02-20T08:11:39.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:39.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:39.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.931 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_22 2021-02-20T08:11:39.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:39.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:39.932 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_22 128M 2021-02-20T08:11:39.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:39.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.933 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:39.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_22 2021-02-20T08:11:39.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:39.936 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:39.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_22 2021-02-20T08:11:40.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:11:40.037 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_22 snapshot 2021-02-20T08:11:40.037 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:40.037 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:40.037 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_22 2021-02-20T08:11:40.038 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:40.038 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_22 snapshot 2021-02-20T08:11:40.628 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:11:40.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:11:40.630 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:11:40.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:11:40.625+0000 7f2012624700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 29584) UID: 1000 2021-02-20T08:11:40.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:11:41.095 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_22 2021-02-20T08:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_22 2021-02-20T08:11:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=13 2021-02-20T08:11:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:11:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 42564608 bytes 2021-02-20T08:11:42.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 43180032 bytes 2021-02-20T08:11:42.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 89569280 bytes 2021-02-20T08:11:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 110608384 bytes 2021-02-20T08:11:43.631 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:11:43.632 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:11:43.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:11:43.633 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:11:43.633 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:11:43.633 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:11:43.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:11:43.636 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T08:11:43.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 182071296 bytes 2021-02-20T08:11:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 233419776 bytes 2021-02-20T08:11:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 313634816 bytes 2021-02-20T08:11:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 340790272 bytes 2021-02-20T08:11:45.414 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 341401600 bytes 2021-02-20T08:11:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 398109696 bytes 2021-02-20T08:11:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 427256832 bytes 2021-02-20T08:11:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 458774528 bytes 2021-02-20T08:11:48.008 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 486803456 bytes 2021-02-20T08:11:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 538254336 bytes 2021-02-20T08:11:49.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 570696704 bytes 2021-02-20T08:11:50.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 631568384 bytes 2021-02-20T08:11:51.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 733414400 bytes 2021-02-20T08:11:51.726 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 756579328 bytes 2021-02-20T08:11:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 809927680 bytes 2021-02-20T08:11:53.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 886556672 bytes 2021-02-20T08:11:53.806 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 912055296 bytes 2021-02-20T08:11:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:54.383 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:54.383 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_23 2021-02-20T08:11:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 snapshot 128M 2021-02-20T08:11:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:54.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_23 2021-02-20T08:11:54.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.385 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:54.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:54.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_23 128M 2021-02-20T08:11:54.386 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_23 2021-02-20T08:11:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:54.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:54.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:54.388 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:54.389 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_23 2021-02-20T08:11:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:11:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_23 snapshot 2021-02-20T08:11:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_23 2021-02-20T08:11:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_23 snapshot 2021-02-20T08:11:54.885 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:11:54.915 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:11:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_23 2021-02-20T08:11:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_23 2021-02-20T08:11:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:11:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=5 2021-02-20T08:11:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:11:54.918 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:11:55.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 39171072 bytes 2021-02-20T08:11:56.191 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 80957440 bytes 2021-02-20T08:11:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 81573888 bytes 2021-02-20T08:11:57.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 151292928 bytes 2021-02-20T08:11:57.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 152458240 bytes 2021-02-20T08:11:57.552 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 153799680 bytes 2021-02-20T08:11:58.086 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 177924096 bytes 2021-02-20T08:11:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 227288064 bytes 2021-02-20T08:11:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 270008320 bytes 2021-02-20T08:11:59.717 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 301582336 bytes 2021-02-20T08:11:59.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:11:59.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:11:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:11:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:11:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_24 2021-02-20T08:11:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 snapshot 128M 2021-02-20T08:11:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_24 2021-02-20T08:11:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:11:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:11:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_24 128M 2021-02-20T08:11:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:11:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:11:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_24 2021-02-20T08:11:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:11:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:11:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:11:59.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:11:59.927 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_24 2021-02-20T08:12:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_24 snapshot 2021-02-20T08:12:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_24 2021-02-20T08:12:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:00.014 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_24 snapshot 2021-02-20T08:12:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_24 2021-02-20T08:12:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_24 2021-02-20T08:12:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=7 2021-02-20T08:12:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:01.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:01.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 15820800 bytes 2021-02-20T08:12:02.192 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 67294208 bytes 2021-02-20T08:12:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 112584704 bytes 2021-02-20T08:12:03.478 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 184505344 bytes 2021-02-20T08:12:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 219545600 bytes 2021-02-20T08:12:04.663 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 294292480 bytes 2021-02-20T08:12:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 312625152 bytes 2021-02-20T08:12:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 353110016 bytes 2021-02-20T08:12:05.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9597815313239273) > weight (0.25) 2021-02-20T08:12:05.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7348320773135263) > weight (0.25) 2021-02-20T08:12:05.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32029379038246775) > weight (0.25) 2021-02-20T08:12:05.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9543380615056579) > weight (0.25) 2021-02-20T08:12:05.946 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T08:12:06.464 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 411490304 bytes 2021-02-20T08:12:06.898 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 427935744 bytes 2021-02-20T08:12:07.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 495266816 bytes 2021-02-20T08:12:08.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:12:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:12:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:12:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_25 2021-02-20T08:12:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_25 snapshot 128M 2021-02-20T08:12:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_25 2021-02-20T08:12:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:12:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_25 128M 2021-02-20T08:12:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_25 2021-02-20T08:12:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:12:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:12:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:12:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_25 2021-02-20T08:12:08.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:08.237 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_25 snapshot 2021-02-20T08:12:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_25 2021-02-20T08:12:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_25 snapshot 2021-02-20T08:12:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_25 2021-02-20T08:12:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_25 2021-02-20T08:12:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:09.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=13 2021-02-20T08:12:09.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:09.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_25 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:09.787 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 33065984 bytes 2021-02-20T08:12:10.208 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 71224320 bytes 2021-02-20T08:12:10.655 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 132291584 bytes 2021-02-20T08:12:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 174561280 bytes 2021-02-20T08:12:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 193501184 bytes 2021-02-20T08:12:12.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 254891008 bytes 2021-02-20T08:12:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 312944640 bytes 2021-02-20T08:12:13.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 341852160 bytes 2021-02-20T08:12:13.507 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 392787968 bytes 2021-02-20T08:12:14.371 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 477657088 bytes 2021-02-20T08:12:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 512406528 bytes 2021-02-20T08:12:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 571531264 bytes 2021-02-20T08:12:16.157 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 617121792 bytes 2021-02-20T08:12:16.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 632544256 bytes 2021-02-20T08:12:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 712666112 bytes 2021-02-20T08:12:18.075 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 809317376 bytes 2021-02-20T08:12:18.646 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 829450240 bytes 2021-02-20T08:12:19.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 885702656 bytes 2021-02-20T08:12:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 910147584 bytes 2021-02-20T08:12:20.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 936990720 bytes 2021-02-20T08:12:20.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 970037248 bytes 2021-02-20T08:12:21.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 1002655744 bytes 2021-02-20T08:12:22.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 1039854592 bytes 2021-02-20T08:12:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:12:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:12:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:12:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_26 2021-02-20T08:12:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_26 snapshot 128M 2021-02-20T08:12:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_26 2021-02-20T08:12:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:12:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_26 128M 2021-02-20T08:12:22.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:22.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_26 2021-02-20T08:12:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:12:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:12:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:12:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_26 2021-02-20T08:12:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_26 snapshot 2021-02-20T08:12:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:22.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_26 2021-02-20T08:12:22.283 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:22.284 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_26 snapshot 2021-02-20T08:12:22.908 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:22.952 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:22.952 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_26 2021-02-20T08:12:22.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:22.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:22.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_26 2021-02-20T08:12:22.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=7 2021-02-20T08:12:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_26 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:24.042 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 69439488 bytes 2021-02-20T08:12:24.498 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 113896448 bytes 2021-02-20T08:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 160160768 bytes 2021-02-20T08:12:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 270381056 bytes 2021-02-20T08:12:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 279365632 bytes 2021-02-20T08:12:26.441 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 284305408 bytes 2021-02-20T08:12:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 301072384 bytes 2021-02-20T08:12:27.374 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 344492032 bytes 2021-02-20T08:12:27.911 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 375938048 bytes 2021-02-20T08:12:28.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 397966336 bytes 2021-02-20T08:12:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 403227648 bytes 2021-02-20T08:12:29.435 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 447870976 bytes 2021-02-20T08:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_27 2021-02-20T08:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_27 snapshot 128M 2021-02-20T08:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_27 2021-02-20T08:12:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:29.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:12:29.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_27 128M 2021-02-20T08:12:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_27 2021-02-20T08:12:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:12:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:12:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:12:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_27 2021-02-20T08:12:30.053 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_27 snapshot 2021-02-20T08:12:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_27 2021-02-20T08:12:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_27 snapshot 2021-02-20T08:12:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_27 2021-02-20T08:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_27 2021-02-20T08:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=14 2021-02-20T08:12:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_27 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 27348992 bytes 2021-02-20T08:12:32.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 64209920 bytes 2021-02-20T08:12:32.833 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 83193856 bytes 2021-02-20T08:12:33.275 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 111638528 bytes 2021-02-20T08:12:33.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 184373248 bytes 2021-02-20T08:12:34.523 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 224243712 bytes 2021-02-20T08:12:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 312468480 bytes 2021-02-20T08:12:35.779 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 344259584 bytes 2021-02-20T08:12:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 370013184 bytes 2021-02-20T08:12:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 383251456 bytes 2021-02-20T08:12:37.288 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 407554048 bytes 2021-02-20T08:12:38.352 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 552608768 bytes 2021-02-20T08:12:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 572316672 bytes 2021-02-20T08:12:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 616784896 bytes 2021-02-20T08:12:40.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 651499520 bytes 2021-02-20T08:12:40.713 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 724076544 bytes 2021-02-20T08:12:41.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 761987072 bytes 2021-02-20T08:12:41.746 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 781586432 bytes 2021-02-20T08:12:42.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 826573824 bytes 2021-02-20T08:12:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 892766208 bytes 2021-02-20T08:12:43.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 951938048 bytes 2021-02-20T08:12:44.316 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2200 IOs, 993990656 bytes 2021-02-20T08:12:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2300 IOs, 1052519424 bytes 2021-02-20T08:12:45.147 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:12:45.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:12:45.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:45.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:12:45.148 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_28 2021-02-20T08:12:45.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_28 snapshot 128M 2021-02-20T08:12:45.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:45.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:45.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_28 2021-02-20T08:12:45.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:45.150 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:12:45.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_28 128M 2021-02-20T08:12:45.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:45.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:45.152 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.152 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_28 2021-02-20T08:12:45.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:12:45.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:12:45.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:12:45.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:45.154 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_28 2021-02-20T08:12:45.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:45.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_28 snapshot 2021-02-20T08:12:45.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_28 2021-02-20T08:12:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_28 snapshot 2021-02-20T08:12:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_28 2021-02-20T08:12:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_28 2021-02-20T08:12:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=11 2021-02-20T08:12:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_28 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 6767616 bytes 2021-02-20T08:12:47.756 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 96196608 bytes 2021-02-20T08:12:48.377 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 140042240 bytes 2021-02-20T08:12:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 169596928 bytes 2021-02-20T08:12:49.223 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 185671680 bytes 2021-02-20T08:12:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 214079488 bytes 2021-02-20T08:12:49.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 229072896 bytes 2021-02-20T08:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 313714688 bytes 2021-02-20T08:12:51.048 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 352124928 bytes 2021-02-20T08:12:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 423911424 bytes 2021-02-20T08:12:52.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 467062784 bytes 2021-02-20T08:12:52.731 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 536355840 bytes 2021-02-20T08:12:53.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 574411776 bytes 2021-02-20T08:12:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 639818752 bytes 2021-02-20T08:12:54.375 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 651204608 bytes 2021-02-20T08:12:54.851 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 661611520 bytes 2021-02-20T08:12:55.405 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 680111104 bytes 2021-02-20T08:12:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 761841664 bytes 2021-02-20T08:12:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 793537536 bytes 2021-02-20T08:12:57.323 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:12:57.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:12:57.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:12:57.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:12:57.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_29 2021-02-20T08:12:57.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_29 snapshot 128M 2021-02-20T08:12:57.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:57.325 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:57.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_29 2021-02-20T08:12:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:12:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_29 128M 2021-02-20T08:12:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:57.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.329 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_29 2021-02-20T08:12:57.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:12:57.330 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:12:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:12:57.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:12:57.334 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_29 2021-02-20T08:12:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:12:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_29 snapshot 2021-02-20T08:12:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_29 2021-02-20T08:12:57.414 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:12:57.414 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_29 snapshot 2021-02-20T08:12:57.919 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:12:57.925 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:12:57.915+0000 7f803c4bb700 -1 reset not still connected to 0x55acb3365190 2021-02-20T08:12:57.926 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:12:57.915+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8caea90 2021-02-20T08:12:57.926 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:12:57.915+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b5cf0 2021-02-20T08:12:57.975 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:12:57.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_29 2021-02-20T08:12:57.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:12:57.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:12:57.976 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_29 2021-02-20T08:12:57.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:12:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=7 2021-02-20T08:12:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:12:57.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_29 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:12:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 61708288 bytes 2021-02-20T08:12:59.676 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 166557696 bytes 2021-02-20T08:13:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 169865216 bytes 2021-02-20T08:13:00.554 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 198488064 bytes 2021-02-20T08:13:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 288020480 bytes 2021-02-20T08:13:01.939 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 359400448 bytes 2021-02-20T08:13:02.641 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 398584832 bytes 2021-02-20T08:13:03.306 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 442982400 bytes 2021-02-20T08:13:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 487212032 bytes 2021-02-20T08:13:04.567 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 524112896 bytes 2021-02-20T08:13:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:13:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:13:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:04.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:13:04.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_30 2021-02-20T08:13:04.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_30 snapshot 128M 2021-02-20T08:13:04.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:04.983 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:04.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_30 2021-02-20T08:13:04.984 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:13:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_30 128M 2021-02-20T08:13:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_30 2021-02-20T08:13:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:13:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:13:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:13:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:04.988 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_30 2021-02-20T08:13:05.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:13:05.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_30 snapshot 2021-02-20T08:13:05.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:05.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:05.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_30 2021-02-20T08:13:05.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:05.113 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_30 snapshot 2021-02-20T08:13:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:13:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:13:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_30 2021-02-20T08:13:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:05.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:05.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_30 2021-02-20T08:13:05.409 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:05.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=6 2021-02-20T08:13:05.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:13:05.410 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_30 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 14836736 bytes 2021-02-20T08:13:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 46400512 bytes 2021-02-20T08:13:06.929 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 69358592 bytes 2021-02-20T08:13:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 160376832 bytes 2021-02-20T08:13:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 211527680 bytes 2021-02-20T08:13:08.571 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 224988160 bytes 2021-02-20T08:13:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 294450176 bytes 2021-02-20T08:13:09.857 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 334081024 bytes 2021-02-20T08:13:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 382255104 bytes 2021-02-20T08:13:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 382866432 bytes 2021-02-20T08:13:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 383469568 bytes 2021-02-20T08:13:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:13:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:13:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:13:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_31 2021-02-20T08:13:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_31 snapshot 128M 2021-02-20T08:13:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_31 2021-02-20T08:13:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:13:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.418 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_31 128M 2021-02-20T08:13:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_31 2021-02-20T08:13:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:13:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:13:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:13:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_31 2021-02-20T08:13:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:13:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_31 snapshot 2021-02-20T08:13:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_31 2021-02-20T08:13:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_31 snapshot 2021-02-20T08:13:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:13:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:13:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_31 2021-02-20T08:13:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_31 2021-02-20T08:13:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:11.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T08:13:11.980 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:13:12.727 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_31 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 25422848 bytes 2021-02-20T08:13:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 88446976 bytes 2021-02-20T08:13:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 153654272 bytes 2021-02-20T08:13:15.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 204289024 bytes 2021-02-20T08:13:15.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 204907520 bytes 2021-02-20T08:13:15.902 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 258771968 bytes 2021-02-20T08:13:16.444 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 324731904 bytes 2021-02-20T08:13:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 366855168 bytes 2021-02-20T08:13:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 404147200 bytes 2021-02-20T08:13:17.934 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 422606848 bytes 2021-02-20T08:13:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 486414336 bytes 2021-02-20T08:13:19.293 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 538425344 bytes 2021-02-20T08:13:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 576765952 bytes 2021-02-20T08:13:20.573 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 633761792 bytes 2021-02-20T08:13:21.490 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 717409280 bytes 2021-02-20T08:13:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:13:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:13:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:13:21.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_32 2021-02-20T08:13:21.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_32 snapshot 128M 2021-02-20T08:13:21.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:21.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_32 2021-02-20T08:13:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:13:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_32 128M 2021-02-20T08:13:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_32 2021-02-20T08:13:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:13:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:13:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:13:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:21.738 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_32 2021-02-20T08:13:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:13:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_32 snapshot 2021-02-20T08:13:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_32 2021-02-20T08:13:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_32 snapshot 2021-02-20T08:13:22.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:13:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:13:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_32 2021-02-20T08:13:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_32 2021-02-20T08:13:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T08:13:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:13:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_32 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 35883008 bytes 2021-02-20T08:13:24.380 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 99362816 bytes 2021-02-20T08:13:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 122936320 bytes 2021-02-20T08:13:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 205595648 bytes 2021-02-20T08:13:26.073 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 261087232 bytes 2021-02-20T08:13:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 325524480 bytes 2021-02-20T08:13:27.644 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 393589760 bytes 2021-02-20T08:13:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 448919552 bytes 2021-02-20T08:13:28.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 516358144 bytes 2021-02-20T08:13:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 551083008 bytes 2021-02-20T08:13:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 632616960 bytes 2021-02-20T08:13:30.494 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 677803008 bytes 2021-02-20T08:13:31.290 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 800015360 bytes 2021-02-20T08:13:32.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:13:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:13:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:13:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_33 2021-02-20T08:13:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_33 snapshot 128M 2021-02-20T08:13:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:32.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_33 2021-02-20T08:13:32.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:32.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:13:32.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_33 128M 2021-02-20T08:13:32.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_33 2021-02-20T08:13:32.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:13:32.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:13:32.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:13:32.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:32.036 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_33 2021-02-20T08:13:32.216 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:13:32.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_33 snapshot 2021-02-20T08:13:32.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_33 2021-02-20T08:13:32.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:32.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_33 snapshot 2021-02-20T08:13:32.916 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:13:32.953 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:13:32.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_33 2021-02-20T08:13:32.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:32.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:32.954 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_33 2021-02-20T08:13:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:13:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:13:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_33 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:33.737 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 99278848 bytes 2021-02-20T08:13:34.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 138430464 bytes 2021-02-20T08:13:34.761 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 191471616 bytes 2021-02-20T08:13:35.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 222876672 bytes 2021-02-20T08:13:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 267681792 bytes 2021-02-20T08:13:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 336598016 bytes 2021-02-20T08:13:36.747 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 408192000 bytes 2021-02-20T08:13:37.533 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 450769920 bytes 2021-02-20T08:13:37.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9865227701441127) > weight (0.25) 2021-02-20T08:13:37.638 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:13:37.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:13:37.639 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:13:37.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:13:37.639+0000 7f2254211700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 26679) UID: 1000 2021-02-20T08:13:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 496221184 bytes 2021-02-20T08:13:38.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:13:38.935 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 550608896 bytes 2021-02-20T08:13:39.561 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 573742080 bytes 2021-02-20T08:13:40.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 602086400 bytes 2021-02-20T08:13:40.640 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:13:40.640 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:13:40.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:13:40.641 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:13:40.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:13:40.642 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:13:40.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:13:40.644 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:13:40.840 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 640458752 bytes 2021-02-20T08:13:41.586 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 679106560 bytes 2021-02-20T08:13:42.009 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 681704448 bytes 2021-02-20T08:13:42.708 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 732810240 bytes 2021-02-20T08:13:43.335 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 793563136 bytes 2021-02-20T08:13:44.407 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 865192960 bytes 2021-02-20T08:13:44.883 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 865803264 bytes 2021-02-20T08:13:44.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:13:44.961 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:13:44.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:13:44.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:13:44.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_34 2021-02-20T08:13:44.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_34 snapshot 128M 2021-02-20T08:13:44.962 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_34 2021-02-20T08:13:44.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:44.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:13:44.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.964 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_34 128M 2021-02-20T08:13:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_34 2021-02-20T08:13:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:13:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:13:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:13:44.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:13:44.967 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_34 2021-02-20T08:13:45.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:13:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_34 snapshot 2021-02-20T08:13:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_34 2021-02-20T08:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_34 snapshot 2021-02-20T08:13:46.045 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:13:46.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:13:46.091 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_34 2021-02-20T08:13:46.092 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:13:46.092 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:13:46.092 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_34 2021-02-20T08:13:46.092 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:13:46.093 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=14 2021-02-20T08:13:46.093 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:13:46.093 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_34 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:13:46.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 35415040 bytes 2021-02-20T08:13:47.184 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 64557056 bytes 2021-02-20T08:13:47.604 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 97304576 bytes 2021-02-20T08:13:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 108947456 bytes 2021-02-20T08:13:48.309 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 138268672 bytes 2021-02-20T08:13:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 160314368 bytes 2021-02-20T08:13:49.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 253901824 bytes 2021-02-20T08:13:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 262666240 bytes 2021-02-20T08:13:50.198 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 282372096 bytes 2021-02-20T08:13:50.965 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 352278528 bytes 2021-02-20T08:13:51.786 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 421237760 bytes 2021-02-20T08:13:51.947 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6497079970212764) > weight (0.25) 2021-02-20T08:13:51.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3309647619815229) > weight (0.25) 2021-02-20T08:13:51.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7466406448259315) > weight (0.25) 2021-02-20T08:13:51.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7929060739586006) > weight (0.25) 2021-02-20T08:13:51.948 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T08:13:52.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 444686336 bytes 2021-02-20T08:13:52.769 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 447048704 bytes 2021-02-20T08:13:53.362 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 508640256 bytes 2021-02-20T08:13:54.413 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 576327680 bytes 2021-02-20T08:13:54.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 604806144 bytes 2021-02-20T08:13:55.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 626266112 bytes 2021-02-20T08:13:56.375 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1800 IOs, 708722688 bytes 2021-02-20T08:13:57.428 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1900 IOs, 819289088 bytes 2021-02-20T08:13:58.332 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2000 IOs, 884649984 bytes 2021-02-20T08:13:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:2100 IOs, 940548096 bytes 2021-02-20T08:14:00.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_35 2021-02-20T08:14:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_35 snapshot 128M 2021-02-20T08:14:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_35 2021-02-20T08:14:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:00.102 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_35 128M 2021-02-20T08:14:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_35 2021-02-20T08:14:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_35 2021-02-20T08:14:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_35 snapshot 2021-02-20T08:14:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_35 2021-02-20T08:14:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_35 snapshot 2021-02-20T08:14:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_35 2021-02-20T08:14:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_35 2021-02-20T08:14:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:14:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_35 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 41447424 bytes 2021-02-20T08:14:02.402 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 125704192 bytes 2021-02-20T08:14:03.360 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 213938176 bytes 2021-02-20T08:14:04.219 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 266989568 bytes 2021-02-20T08:14:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 324666368 bytes 2021-02-20T08:14:05.518 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 358607872 bytes 2021-02-20T08:14:06.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 379319296 bytes 2021-02-20T08:14:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 406717440 bytes 2021-02-20T08:14:07.224 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 450682880 bytes 2021-02-20T08:14:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 465618944 bytes 2021-02-20T08:14:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 504680448 bytes 2021-02-20T08:14:09.307 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 613145600 bytes 2021-02-20T08:14:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 663054336 bytes 2021-02-20T08:14:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 700067840 bytes 2021-02-20T08:14:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 756743168 bytes 2021-02-20T08:14:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 800476160 bytes 2021-02-20T08:14:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 847960064 bytes 2021-02-20T08:14:13.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:13.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:13.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:13.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:13.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_36 2021-02-20T08:14:13.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_36 snapshot 128M 2021-02-20T08:14:13.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:13.030 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:13.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_36 2021-02-20T08:14:13.031 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_36 128M 2021-02-20T08:14:13.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_36 2021-02-20T08:14:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:13.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:13.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:13.034 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:13.035 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_36 2021-02-20T08:14:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_36 snapshot 2021-02-20T08:14:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_36 2021-02-20T08:14:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_36 snapshot 2021-02-20T08:14:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_36 2021-02-20T08:14:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:14.060 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:14.060 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_36 2021-02-20T08:14:14.060 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:14.061 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=9 2021-02-20T08:14:14.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:14.062 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_36 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 22771712 bytes 2021-02-20T08:14:15.639 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 105918464 bytes 2021-02-20T08:14:16.064 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 143296512 bytes 2021-02-20T08:14:16.538 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 186597376 bytes 2021-02-20T08:14:16.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 208604160 bytes 2021-02-20T08:14:17.379 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 270902272 bytes 2021-02-20T08:14:17.923 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 306798592 bytes 2021-02-20T08:14:18.438 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 358396928 bytes 2021-02-20T08:14:19.265 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 471362560 bytes 2021-02-20T08:14:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 522040320 bytes 2021-02-20T08:14:20.370 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 552377344 bytes 2021-02-20T08:14:20.712 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 585627648 bytes 2021-02-20T08:14:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 656568320 bytes 2021-02-20T08:14:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 661768192 bytes 2021-02-20T08:14:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 675660800 bytes 2021-02-20T08:14:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 710641664 bytes 2021-02-20T08:14:23.065 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_37 2021-02-20T08:14:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_37 snapshot 128M 2021-02-20T08:14:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_37 2021-02-20T08:14:23.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:23.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:23.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_37 128M 2021-02-20T08:14:23.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:23.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_37 2021-02-20T08:14:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:23.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:23.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:23.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:23.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_37 2021-02-20T08:14:23.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:23.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_37 snapshot 2021-02-20T08:14:23.217 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:23.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:23.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_37 2021-02-20T08:14:23.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:23.218 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_37 snapshot 2021-02-20T08:14:24.026 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:24.068 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:24.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_37 2021-02-20T08:14:24.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:24.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:24.069 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_37 2021-02-20T08:14:24.070 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=6 2021-02-20T08:14:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_37 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:25.032 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 73861120 bytes 2021-02-20T08:14:25.384 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 88517632 bytes 2021-02-20T08:14:26.033 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 204471296 bytes 2021-02-20T08:14:26.600 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 248853504 bytes 2021-02-20T08:14:26.926 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 251582464 bytes 2021-02-20T08:14:27.501 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 302903296 bytes 2021-02-20T08:14:28.088 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 339186688 bytes 2021-02-20T08:14:28.698 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 377100288 bytes 2021-02-20T08:14:29.354 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 421786624 bytes 2021-02-20T08:14:30.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:30.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:30.076 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_38 2021-02-20T08:14:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_38 snapshot 128M 2021-02-20T08:14:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_38 2021-02-20T08:14:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:30.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:30.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.079 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_38 128M 2021-02-20T08:14:30.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:30.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:30.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.080 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_38 2021-02-20T08:14:30.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:30.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:30.081 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:30.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:30.082 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_38 2021-02-20T08:14:30.295 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_38 snapshot 2021-02-20T08:14:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_38 2021-02-20T08:14:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_38 snapshot 2021-02-20T08:14:31.052 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:31.097 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:31.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_38 2021-02-20T08:14:31.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:31.098 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:31.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_38 2021-02-20T08:14:31.099 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:31.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:14:31.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:31.100 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_38 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:31.831 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 33350656 bytes 2021-02-20T08:14:32.204 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 54356992 bytes 2021-02-20T08:14:32.607 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 90486784 bytes 2021-02-20T08:14:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 111110144 bytes 2021-02-20T08:14:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 136898560 bytes 2021-02-20T08:14:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 209740800 bytes 2021-02-20T08:14:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 245167104 bytes 2021-02-20T08:14:35.567 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 297592832 bytes 2021-02-20T08:14:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 326943744 bytes 2021-02-20T08:14:36.616 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T08:14:36.609+0000 7fc273bc1700 -1 reset not still connected to 0x56116c1c1450 2021-02-20T08:14:36.617 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T08:14:36.609+0000 7fc273bc1700 -1 reset not still connected to 0x56116c1c2e50 2021-02-20T08:14:36.700 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 375247872 bytes 2021-02-20T08:14:37.381 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 420466688 bytes 2021-02-20T08:14:38.453 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 511341568 bytes 2021-02-20T08:14:39.574 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 606168064 bytes 2021-02-20T08:14:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 668138496 bytes 2021-02-20T08:14:41.176 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 727985152 bytes 2021-02-20T08:14:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 833427456 bytes 2021-02-20T08:14:43.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:43.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:43.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:43.107 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:43.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_39 2021-02-20T08:14:43.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_39 snapshot 128M 2021-02-20T08:14:43.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:43.108 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_39 2021-02-20T08:14:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_39 128M 2021-02-20T08:14:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:43.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:43.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_39 2021-02-20T08:14:43.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_39 2021-02-20T08:14:43.212 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:43.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_39 snapshot 2021-02-20T08:14:43.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:43.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:43.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_39 2021-02-20T08:14:43.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:43.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_39 snapshot 2021-02-20T08:14:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:44.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:44.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_39 2021-02-20T08:14:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_39 2021-02-20T08:14:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:14:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_39 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:45.324 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 85289984 bytes 2021-02-20T08:14:45.947 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 142296064 bytes 2021-02-20T08:14:46.356 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 165474304 bytes 2021-02-20T08:14:46.733 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 192009216 bytes 2021-02-20T08:14:47.117 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 213511168 bytes 2021-02-20T08:14:47.724 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 284674048 bytes 2021-02-20T08:14:47.985 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 285254656 bytes 2021-02-20T08:14:48.326 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 305586176 bytes 2021-02-20T08:14:48.742 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 328239104 bytes 2021-02-20T08:14:49.340 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 410462208 bytes 2021-02-20T08:14:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 428554240 bytes 2021-02-20T08:14:50.631 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 515905536 bytes 2021-02-20T08:14:51.205 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 585057280 bytes 2021-02-20T08:14:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 637588480 bytes 2021-02-20T08:14:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:14:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:14:52.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:14:52.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:14:52.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_40 2021-02-20T08:14:52.169 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_40 snapshot 128M 2021-02-20T08:14:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_40 2021-02-20T08:14:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:14:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_40 128M 2021-02-20T08:14:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_40 2021-02-20T08:14:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:14:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:14:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:14:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:14:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_40 2021-02-20T08:14:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:14:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_40 snapshot 2021-02-20T08:14:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_40 2021-02-20T08:14:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:14:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_40 snapshot 2021-02-20T08:14:53.085 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:14:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:14:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_40 2021-02-20T08:14:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:14:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:14:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_40 2021-02-20T08:14:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:14:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=8 2021-02-20T08:14:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:14:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_40 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:14:53.780 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 29386752 bytes 2021-02-20T08:14:53.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9458437420268428) > weight (0.25) 2021-02-20T08:14:53.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5212426564793986) > weight (0.25) 2021-02-20T08:14:53.950 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:14:53.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:14:53.951 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:14:53.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:14:53.949+0000 7f0748eda700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 30582) UID: 1000 2021-02-20T08:14:54.182 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 49785856 bytes 2021-02-20T08:14:54.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:14:54.635 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 90639360 bytes 2021-02-20T08:14:55.029 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 116236288 bytes 2021-02-20T08:14:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 171609088 bytes 2021-02-20T08:14:56.570 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 253654016 bytes 2021-02-20T08:14:57.226 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 277893120 bytes 2021-02-20T08:14:57.695 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 284687360 bytes 2021-02-20T08:14:58.485 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 350578688 bytes 2021-02-20T08:14:59.122 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 370798592 bytes 2021-02-20T08:14:59.859 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 412538880 bytes 2021-02-20T08:15:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 422937600 bytes 2021-02-20T08:15:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ error_code=124 2021-02-20T08:15:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set -e 2021-02-20T08:15:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ 124 -eq 124 ] 2021-02-20T08:15:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ return 0 2021-02-20T08:15:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ image=image_41 2021-02-20T08:15:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image_and_enable_mirror cluster2 mirror image_41 snapshot 128M 2021-02-20T08:15:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:15:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:15:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_41 2021-02-20T08:15:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:15:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n snapshot ] 2021-02-20T08:15:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ create_image cluster2 mirror image_41 128M 2021-02-20T08:15:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:15:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:15:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_41 2021-02-20T08:15:01.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local size=128 2021-02-20T08:15:01.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n 128M ] 2021-02-20T08:15:01.137 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ size=128M 2021-02-20T08:15:01.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ shift 2021-02-20T08:15:01.138 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image_41 2021-02-20T08:15:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ image = image ] 2021-02-20T08:15:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ enable_mirror cluster2 mirror image_41 snapshot 2021-02-20T08:15:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:15:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:15:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_41 2021-02-20T08:15:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local mode=snapshot 2021-02-20T08:15:01.215 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_41 snapshot 2021-02-20T08:15:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:Mirroring enabled 2021-02-20T08:15:02.020 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ [ -n ] 2021-02-20T08:15:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ stress_write_image cluster2 mirror image_41 2021-02-20T08:15:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local cluster=cluster2 2021-02-20T08:15:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local pool=mirror 2021-02-20T08:15:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local image=image_41 2021-02-20T08:15:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-02-20T08:15:02.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ local duration=12 2021-02-20T08:15:02.022 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ set +e 2021-02-20T08:15:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_41 --debug-rbd=20 --debug-journaler=20 2021-02-20T08:15:02.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:100 IOs, 36358144 bytes 2021-02-20T08:15:02.952 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:15:02.952 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:15:02.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:15:02.953 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:15:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:15:02.954 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:15:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:15:02.957 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T08:15:03.542 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:200 IOs, 57474048 bytes 2021-02-20T08:15:04.160 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:300 IOs, 127191040 bytes 2021-02-20T08:15:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:400 IOs, 168741888 bytes 2021-02-20T08:15:05.136 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:500 IOs, 218473472 bytes 2021-02-20T08:15:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:600 IOs, 257697792 bytes 2021-02-20T08:15:06.764 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:700 IOs, 378942464 bytes 2021-02-20T08:15:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:800 IOs, 418686976 bytes 2021-02-20T08:15:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:900 IOs, 444846080 bytes 2021-02-20T08:15:08.637 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1000 IOs, 484783104 bytes 2021-02-20T08:15:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1100 IOs, 572240896 bytes 2021-02-20T08:15:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1200 IOs, 645619712 bytes 2021-02-20T08:15:11.111 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1300 IOs, 692297728 bytes 2021-02-20T08:15:11.783 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1400 IOs, 730109952 bytes 2021-02-20T08:15:12.693 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1500 IOs, 806090752 bytes 2021-02-20T08:15:13.318 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1600 IOs, 844779520 bytes 2021-02-20T08:15:13.809 INFO:tasks.workunit.cluster1.client.mirror.smithi114.stdout:1700 IOs, 881413120 bytes 2021-02-20T08:15:14.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.785382288134187) > weight (0.25) 2021-02-20T08:15:14.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7708821326953332) > weight (0.25) 2021-02-20T08:15:14.646 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:15:14.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:15:14.647 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:15:14.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T08:15:14.644+0000 7fb41b0b1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 30492) UID: 1000 2021-02-20T08:15:15.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:15:16.648 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:15:16.649 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:15:16.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:15:16.650 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:15:16.650 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:15:16.650 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:15:16.652 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:15:16.653 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T08:15:41.376 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T08:15:41.369+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c9520 2021-02-20T08:16:05.958 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:16:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:16:05.960 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:16:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:16:05.955+0000 7f2bc6d66700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 27807) UID: 1000 2021-02-20T08:16:06.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:16:10.961 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:16:10.961 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:16:10.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:16:10.963 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:16:10.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:16:10.964 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:16:10.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:16:10.968 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T08:16:46.654 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:16:46.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:16:46.655 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:16:46.655 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:16:46.656 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:16:46.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:16:46.654+0000 7f84a35a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 31212) UID: 1000 2021-02-20T08:16:47.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:16:52.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:16:52.758 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:16:52.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:16:52.759 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:16:52.761 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:16:52.762 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T08:17:22.969 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3053785662594327) > weight (0.25) 2021-02-20T08:17:22.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3487080422042528) > weight (0.25) 2021-02-20T08:17:22.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6909068145487341) > weight (0.25) 2021-02-20T08:17:22.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.691393136241023) > weight (0.25) 2021-02-20T08:17:22.971 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T08:17:52.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7374044591486754) > weight (0.25) 2021-02-20T08:17:52.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5276764315771103) > weight (0.25) 2021-02-20T08:17:52.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8515739100424372) > weight (0.25) 2021-02-20T08:17:52.767 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:17:52.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:17:52.770 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:17:52.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T08:17:52.765+0000 7f92806b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 19099) UID: 1000 2021-02-20T08:17:53.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:17:53.771 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:17:53.771 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:17:53.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:17:53.772 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:17:53.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:17:53.773 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:17:53.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:17:53.776 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T08:18:53.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7786579422380467) > weight (0.25) 2021-02-20T08:18:53.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7386823528185562) > weight (0.25) 2021-02-20T08:18:53.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42696059610951065) > weight (0.25) 2021-02-20T08:18:53.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8106994269067386) > weight (0.25) 2021-02-20T08:18:53.778 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T08:19:18.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9880854232837419) > weight (0.25) 2021-02-20T08:19:18.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6174662360541269) > weight (0.25) 2021-02-20T08:19:18.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2648983235028054) > weight (0.25) 2021-02-20T08:19:18.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4258100568045605) > weight (0.25) 2021-02-20T08:19:18.973 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T08:20:15.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.605585793578665) > weight (0.25) 2021-02-20T08:20:15.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7237383204213279) > weight (0.25) 2021-02-20T08:20:15.823 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:20:15.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:20:15.825 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:20:15.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T08:20:15.822+0000 7f4529c82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 32662) UID: 1000 2021-02-20T08:20:16.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:20:23.825 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:20:23.831 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:20:23.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:20:23.832 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:20:23.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:20:23.833 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:20:23.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:20:23.836 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T08:20:41.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.635938895103296) > weight (0.25) 2021-02-20T08:20:41.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6993453388727597) > weight (0.25) 2021-02-20T08:20:41.974 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6186430824007482) > weight (0.25) 2021-02-20T08:20:41.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6447130484584825) > weight (0.25) 2021-02-20T08:20:41.975 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T08:21:44.836 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:21:44.837 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:21:44.838 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:21:44.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:21:44.836+0000 7fb1d8d8b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32852) UID: 1000 2021-02-20T08:21:45.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:21:47.839 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:21:47.844 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:21:47.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:21:47.845 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:21:47.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:21:47.846 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:21:47.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:21:47.849 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:22:00.976 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:22:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:22:00.979 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:22:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:22:00.976+0000 7f57b536c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 32753) UID: 1000 2021-02-20T08:22:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:22:08.980 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:22:08.981 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:22:08.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:22:08.982 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:22:08.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:22:08.983 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:22:08.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:22:08.986 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T08:23:07.850 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2819527152978968) > weight (0.25) 2021-02-20T08:23:07.851 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3957880034377603) > weight (0.25) 2021-02-20T08:23:07.851 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.757417749747362) > weight (0.25) 2021-02-20T08:23:07.851 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6669894986656404) > weight (0.25) 2021-02-20T08:23:07.852 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T08:23:19.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9577067608832737) > weight (0.25) 2021-02-20T08:23:19.987 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4172287878129466) > weight (0.25) 2021-02-20T08:23:19.987 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:23:19.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:23:19.989 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:23:19.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:23:19.986+0000 7ff8ab31b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 32561) UID: 1000 2021-02-20T08:23:20.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:23:22.990 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:23:22.991 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:23:22.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:23:22.992 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:23:22.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:23:22.993 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:23:22.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:23:22.996 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T08:23:25.238 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:23:25.234+0000 7f803c4bb700 -1 reset not still connected to 0x55acb0d02b10 2021-02-20T08:23:25.238 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:23:25.234+0000 7f803c4bb700 -1 reset not still connected to 0x55acb3362410 2021-02-20T08:24:09.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6382775831777664) > weight (0.25) 2021-02-20T08:24:09.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.48081628665879184) > weight (0.25) 2021-02-20T08:24:09.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4496503090009617) > weight (0.25) 2021-02-20T08:24:09.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6060763467702097) > weight (0.25) 2021-02-20T08:24:09.855 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T08:24:52.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.95217190060333) > weight (0.25) 2021-02-20T08:24:52.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6033079332490995) > weight (0.25) 2021-02-20T08:24:52.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3489285221487194) > weight (0.25) 2021-02-20T08:24:52.998 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:24:52.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:24:52.999 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:24:53.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:24:52.996+0000 7f215665b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 24467) UID: 1000 2021-02-20T08:24:53.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:24:56.000 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:24:56.000 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:24:56.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:24:56.001 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:24:56.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:24:56.002 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:24:56.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:24:56.005 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T08:25:31.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9357231981184375) > weight (0.25) 2021-02-20T08:25:31.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9300355905644808) > weight (0.25) 2021-02-20T08:25:31.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7618467197005782) > weight (0.25) 2021-02-20T08:25:31.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7979116995192588) > weight (0.25) 2021-02-20T08:25:31.858 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:26:03.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47821926419326954) > weight (0.25) 2021-02-20T08:26:03.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5141399773153534) > weight (0.25) 2021-02-20T08:26:03.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.31830636777295496) > weight (0.25) 2021-02-20T08:26:03.008 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:26:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:26:03.009 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:26:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:26:03.007+0000 7fd9f099c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33425) UID: 1000 2021-02-20T08:26:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:26:12.010 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:26:12.012 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:26:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:26:12.013 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:26:12.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:26:12.014 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:26:12.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:26:12.017 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:27:05.859 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:27:05.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:27:05.862 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:27:05.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:27:05.857+0000 7fd996095700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33135) UID: 1000 2021-02-20T08:27:06.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:27:10.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:27:10.864 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:27:10.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:27:10.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:27:10.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:27:10.865 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:27:10.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:27:10.868 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T08:27:32.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2970370968656597) > weight (0.25) 2021-02-20T08:27:32.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.291964029116224) > weight (0.25) 2021-02-20T08:27:32.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8774175040234892) > weight (0.25) 2021-02-20T08:27:32.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3358725726663253) > weight (0.25) 2021-02-20T08:27:32.019 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:27:38.483 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:27:38.477+0000 7f803c4bb700 -1 reset not still connected to 0x55acb2f19330 2021-02-20T08:27:38.483 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:27:38.477+0000 7f803c4bb700 -1 reset not still connected to 0x55acb5830f70 2021-02-20T08:27:38.483 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:27:38.477+0000 7f803c4bb700 -1 reset not still connected to 0x55acb7d359b0 2021-02-20T08:28:35.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2563980762760487) > weight (0.25) 2021-02-20T08:28:35.870 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:28:35.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:28:35.872 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:28:35.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:28:35.867+0000 7f062d1b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 31971) UID: 1000 2021-02-20T08:28:36.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:28:40.872 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:28:40.873 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:28:40.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:28:40.874 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:28:40.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:28:40.874 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:28:40.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:28:40.877 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T08:28:52.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4213638526263682) > weight (0.25) 2021-02-20T08:28:52.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6486559821743818) > weight (0.25) 2021-02-20T08:28:52.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8039577851156384) > weight (0.25) 2021-02-20T08:28:52.022 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:28:52.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:28:52.023 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:28:52.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:28:52.019+0000 7f80b589a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33530) UID: 1000 2021-02-20T08:28:52.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:28:57.024 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:28:57.025 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:28:57.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:28:57.025 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:28:57.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:28:57.026 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:28:57.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:28:57.029 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T08:29:53.879 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:29:53.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:29:53.880 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:29:53.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:29:53.878+0000 7f47bf783700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33629) UID: 1000 2021-02-20T08:29:54.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:30:00.881 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:30:00.882 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:30:00.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:30:00.883 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:30:00.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:30:00.883 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:30:00.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:30:00.886 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T08:30:51.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9815602307375747) > weight (0.25) 2021-02-20T08:30:51.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7880917924679904) > weight (0.25) 2021-02-20T08:30:51.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5904231715270628) > weight (0.25) 2021-02-20T08:30:51.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6637235352236622) > weight (0.25) 2021-02-20T08:30:51.032 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T08:31:29.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6944007919596568) > weight (0.25) 2021-02-20T08:31:29.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9409791380435397) > weight (0.25) 2021-02-20T08:31:29.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5751219036516225) > weight (0.25) 2021-02-20T08:31:29.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6215896692226112) > weight (0.25) 2021-02-20T08:31:29.889 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T08:32:04.034 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:32:04.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:32:04.036 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:32:04.036 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:32:04.037 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:32:04.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:32:04.035+0000 7f382ae6f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33226) UID: 1000 2021-02-20T08:32:04.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:32:10.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:32:10.139 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:32:10.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:32:10.140 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:32:10.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:32:10.147 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T08:32:31.890 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:32:31.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:32:31.891 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:32:31.892 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:32:31.890+0000 7fd7980f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33910) UID: 1000 2021-02-20T08:32:32.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:32:38.892 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:32:38.893 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:32:38.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:32:38.893 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:32:38.894 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:32:38.894 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:32:38.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:32:38.900 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T08:33:27.148 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6210786578249751) > weight (0.25) 2021-02-20T08:33:27.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5058688127778967) > weight (0.25) 2021-02-20T08:33:27.149 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:33:27.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:33:27.151 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:33:27.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:33:27.149+0000 7f984920c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33326) UID: 1000 2021-02-20T08:33:27.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:33:33.151 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:33:33.152 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:33:33.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:33:33.153 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:33:33.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:33:33.153 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:33:33.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:33:33.156 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T08:34:30.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7568642714759376) > weight (0.25) 2021-02-20T08:34:30.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8791499995190156) > weight (0.25) 2021-02-20T08:34:30.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7260850222536127) > weight (0.25) 2021-02-20T08:34:30.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7346970115442552) > weight (0.25) 2021-02-20T08:34:30.902 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T08:35:26.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7979916570477901) > weight (0.25) 2021-02-20T08:35:26.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7530039169230938) > weight (0.25) 2021-02-20T08:35:26.159 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:35:26.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:35:26.160 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:35:26.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:35:26.159+0000 7fd930a97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34195) UID: 1000 2021-02-20T08:35:26.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:35:27.161 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:35:27.162 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:35:27.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:35:27.163 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:35:27.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:35:27.163 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:35:27.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:35:27.167 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T08:35:56.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5961094702390771) > weight (0.25) 2021-02-20T08:35:56.904 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:35:56.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:35:56.905 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:35:56.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:35:56.902+0000 7f13a10f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 33720) UID: 1000 2021-02-20T08:35:57.307 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:36:03.907 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:36:03.907 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:36:03.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:36:03.908 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:36:03.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:36:03.909 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:36:03.912 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:36:03.912 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T08:36:34.173 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:36:34.169+0000 7f803c4bb700 -1 reset not still connected to 0x55aca6c5bb50 2021-02-20T08:36:34.173 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:36:34.169+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896dad0 2021-02-20T08:37:03.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3144804041544548) > weight (0.25) 2021-02-20T08:37:03.173 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:37:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:37:03.175 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:37:03.175 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:37:03.176 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:37:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T08:37:03.172+0000 7f495c40a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 22394) UID: 1000 2021-02-20T08:37:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:37:09.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:37:09.278 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:37:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:37:09.279 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:37:09.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:37:09.282 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T08:37:50.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5328678703464592) > weight (0.25) 2021-02-20T08:37:50.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5835492826497555) > weight (0.25) 2021-02-20T08:37:50.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7175015482129049) > weight (0.25) 2021-02-20T08:37:50.914 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:37:50.915 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:37:50.916 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T08:37:50.916 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:37:50.916 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:37:50.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T08:37:50.911+0000 7eff1a3af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 32947) UID: 1000 2021-02-20T08:37:51.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:37:57.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:37:57.019 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:37:57.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:37:57.019 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:37:57.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:37:57.022 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T08:38:09.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2616581369925678) > weight (0.25) 2021-02-20T08:38:09.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9135216359519583) > weight (0.25) 2021-02-20T08:38:09.284 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:38:09.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:38:09.285 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:38:09.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:38:09.283+0000 7fa0ae7ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34297) UID: 1000 2021-02-20T08:38:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:38:10.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.515+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9ab400 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T08:38:10.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.515+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9abe00 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T08:38:10.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac580 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T08:38:10.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac800 [3/91cf5a1a-070a-45f0-9e15-e0e31870c4e1] start: already running 2021-02-20T08:38:10.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9ab680 [3/a2357b67-5e9d-4bab-b496-4a5956203c0c] start: already running 2021-02-20T08:38:10.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cb0e68580 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T08:38:10.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac080 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T08:38:10.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9abb80 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T08:38:10.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9aca80 [3/e2100c63-dab1-4809-b5df-77d32cf43dd0] start: already running 2021-02-20T08:38:10.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.519+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac300 [3/fc56e3ca-8114-4f4e-9bba-183b58b29c89] start: already running 2021-02-20T08:38:10.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ab400 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T08:38:10.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9abe00 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T08:38:10.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac580 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T08:38:10.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac800 [3/91cf5a1a-070a-45f0-9e15-e0e31870c4e1] start: already running 2021-02-20T08:38:10.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cac9ab680 [3/a2357b67-5e9d-4bab-b496-4a5956203c0c] start: already running 2021-02-20T08:38:10.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdee7e4700 -1 rbd::mirror::ImageReplayer: 0x557cb0e68580 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T08:38:10.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac080 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T08:38:10.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.539+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9abb80 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T08:38:10.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.555+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9aca80 [3/e2100c63-dab1-4809-b5df-77d32cf43dd0] start: already running 2021-02-20T08:38:10.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.555+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ab900 [3/eb5ee405-f45d-4109-b887-4acc3932b673] start: already running 2021-02-20T08:38:10.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:38:10.555+0000 7efdedfe3700 -1 rbd::mirror::ImageReplayer: 0x557cac9ac300 [3/fc56e3ca-8114-4f4e-9bba-183b58b29c89] start: already running 2021-02-20T08:38:18.286 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:38:18.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:38:18.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:38:18.287 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:38:18.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:38:18.288 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:38:18.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:38:18.291 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T08:39:34.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6399133133717217) > weight (0.25) 2021-02-20T08:39:34.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5548021441265989) > weight (0.25) 2021-02-20T08:39:34.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6745235350895789) > weight (0.25) 2021-02-20T08:39:34.294 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:39:34.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:39:34.295 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:39:34.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:39:34.293+0000 7efdeefe5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33811) UID: 1000 2021-02-20T08:39:34.406 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:39:34.401+0000 7f803c4bb700 -1 reset not still connected to 0x55aca6c5bb50 2021-02-20T08:39:34.407 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:39:34.405+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896dad0 2021-02-20T08:39:34.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:39:37.296 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:39:37.297 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:39:37.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:39:37.297 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:39:37.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:39:37.298 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:39:37.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:39:37.301 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T08:39:40.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9456944857867075) > weight (0.25) 2021-02-20T08:39:40.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4141858489947736) > weight (0.25) 2021-02-20T08:39:40.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5824137093507817) > weight (0.25) 2021-02-20T08:39:40.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8042332031222189) > weight (0.25) 2021-02-20T08:39:40.024 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T08:39:48.548 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:39:48.541+0000 7f803c4bb700 -1 reset not still connected to 0x55acb4af8f70 2021-02-20T08:39:48.549 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:39:48.545+0000 7f803c4bb700 -1 reset not still connected to 0x55acb4afb190 2021-02-20T08:39:48.549 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T08:39:48.545+0000 7f803c4bb700 -1 reset not still connected to 0x55acb4afb400 2021-02-20T08:40:44.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47603343779265184) > weight (0.25) 2021-02-20T08:40:44.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8864691006075387) > weight (0.25) 2021-02-20T08:40:44.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8417895347819094) > weight (0.25) 2021-02-20T08:40:44.313 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:40:44.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:40:44.314 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:40:44.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:40:44.316+0000 7f95cafe3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34780) UID: 1000 2021-02-20T08:40:44.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:40:52.316 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:40:52.316 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:40:52.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:40:52.317 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:40:52.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:40:52.318 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:40:52.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:40:52.321 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T08:41:01.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7877682357244905) > weight (0.25) 2021-02-20T08:41:01.026 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:41:01.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:41:01.027 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:41:01.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:41:01.023+0000 7fe274092700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34396) UID: 1000 2021-02-20T08:41:01.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:41:03.027 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:41:03.028 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:41:03.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:41:03.029 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:41:03.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:41:03.029 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:41:03.032 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:41:03.032 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T08:42:34.322 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40370777959475457) > weight (0.25) 2021-02-20T08:42:34.329 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5309491451754412) > weight (0.25) 2021-02-20T08:42:34.329 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:42:34.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:42:34.330 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:42:34.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:42:34.329+0000 7fa442859700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34680) UID: 1000 2021-02-20T08:42:34.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:42:43.331 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:42:43.332 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:42:43.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:42:43.333 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:42:43.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:42:43.334 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:42:43.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:42:43.339 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T08:42:49.033 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:42:49.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:42:49.034 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:42:49.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:42:49.033+0000 7fe5ad36e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 34104) UID: 1000 2021-02-20T08:42:49.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:42:56.035 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:42:56.036 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:42:56.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:42:56.037 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:42:56.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:42:56.039 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:42:56.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:42:56.048 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T08:44:09.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31384987219133464) > weight (0.25) 2021-02-20T08:44:09.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.94689948307142) > weight (0.25) 2021-02-20T08:44:09.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.770087430516784) > weight (0.25) 2021-02-20T08:44:09.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.27118803050246976) > weight (0.25) 2021-02-20T08:44:09.341 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T08:44:16.049 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43375350384888156) > weight (0.25) 2021-02-20T08:44:16.050 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3100132853741503) > weight (0.25) 2021-02-20T08:44:16.050 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4161154388081604) > weight (0.25) 2021-02-20T08:44:16.050 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:44:16.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:44:16.051 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:44:16.052 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T08:44:16.047+0000 7fa5e301b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 34589) UID: 1000 2021-02-20T08:44:16.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:44:21.053 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:44:21.053 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:44:21.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:44:21.054 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:44:21.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:44:21.055 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:44:21.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:44:21.057 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T08:45:32.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.90835847252629) > weight (0.25) 2021-02-20T08:45:32.060 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7867888344117616) > weight (0.25) 2021-02-20T08:45:32.060 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:45:32.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:45:32.063 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T08:45:32.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T08:45:32.062+0000 7fb91a9cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33040) UID: 1000 2021-02-20T08:45:32.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:45:34.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.915336134099421) > weight (0.25) 2021-02-20T08:45:34.342 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T08:45:34.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T08:45:34.343 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T08:45:34.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T08:45:34.341+0000 7f3ae089a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34490) UID: 1000 2021-02-20T08:45:34.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:45:37.065 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:45:37.065 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:45:37.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:45:37.066 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:45:37.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:45:37.067 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:45:37.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:45:37.070 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T08:45:40.344 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T08:45:40.344 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:45:40.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:45:40.345 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T08:45:40.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T08:45:40.346 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T08:45:40.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T08:45:40.348 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T08:46:51.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.970505242502244) > weight (0.25) 2021-02-20T08:46:51.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36211083813019074) > weight (0.25) 2021-02-20T08:46:51.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.827616923913346) > weight (0.25) 2021-02-20T08:46:51.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7504343779161263) > weight (0.25) 2021-02-20T08:46:51.074 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T08:47:36.350 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33634366573787045) > weight (0.25) 2021-02-20T08:47:36.350 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.798795964569682) > weight (0.25) 2021-02-20T08:47:36.351 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:47:36.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:47:36.352 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T08:47:36.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:47:36.347+0000 7f02922b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35073) UID: 1000 2021-02-20T08:47:36.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:47:39.353 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:47:39.353 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:47:39.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:47:39.354 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:47:39.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:47:39.355 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:47:39.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:47:39.358 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T08:48:30.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6321326100960398) > weight (0.25) 2021-02-20T08:48:30.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.273425644558305) > weight (0.25) 2021-02-20T08:48:30.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42368973923632625) > weight (0.25) 2021-02-20T08:48:30.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9284645947909643) > weight (0.25) 2021-02-20T08:48:30.084 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T08:49:31.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25498635848163076) > weight (0.25) 2021-02-20T08:49:31.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4265717707764156) > weight (0.25) 2021-02-20T08:49:31.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46395137154478616) > weight (0.25) 2021-02-20T08:49:31.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.584429030618215) > weight (0.25) 2021-02-20T08:49:31.361 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T08:50:05.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6874363391916131) > weight (0.25) 2021-02-20T08:50:05.085 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:50:05.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:50:05.087 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:50:05.089 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:50:05.084+0000 7f0acc109700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34979) UID: 1000 2021-02-20T08:50:05.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:50:14.088 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:50:14.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:50:14.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:50:14.090 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:50:14.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:50:14.091 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:50:14.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:50:14.094 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T08:50:53.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8761481987467801) > weight (0.25) 2021-02-20T08:50:53.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.942050360628777) > weight (0.25) 2021-02-20T08:50:53.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3622316323206234) > weight (0.25) 2021-02-20T08:50:53.362 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:50:53.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:50:53.363 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T08:50:53.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:50:53.363+0000 7f052e5c1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 34880) UID: 1000 2021-02-20T08:50:53.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:50:54.364 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:50:54.365 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:50:54.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:50:54.366 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:50:54.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:50:54.366 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:50:54.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:50:54.369 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T08:52:02.094 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T08:52:02.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T08:52:02.096 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T08:52:02.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T08:52:02.093+0000 7f63ad54a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35172) UID: 1000 2021-02-20T08:52:02.498 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:52:10.097 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T08:52:10.098 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:52:10.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:52:10.098 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T08:52:10.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T08:52:10.099 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T08:52:10.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T08:52:10.103 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T08:52:28.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6106429381065065) > weight (0.25) 2021-02-20T08:52:28.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7575809867658292) > weight (0.25) 2021-02-20T08:52:28.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9478568178026622) > weight (0.25) 2021-02-20T08:52:28.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6698280485614895) > weight (0.25) 2021-02-20T08:52:28.371 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T08:53:48.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41658166738994373) > weight (0.25) 2021-02-20T08:53:48.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9717150522410238) > weight (0.25) 2021-02-20T08:53:48.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7431233048214428) > weight (0.25) 2021-02-20T08:53:48.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.27624002957187543) > weight (0.25) 2021-02-20T08:53:48.105 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T08:54:01.372 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8616622048504304) > weight (0.25) 2021-02-20T08:54:01.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4411925597295079) > weight (0.25) 2021-02-20T08:54:01.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6118296341920358) > weight (0.25) 2021-02-20T08:54:01.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6250737316443022) > weight (0.25) 2021-02-20T08:54:01.373 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T08:55:20.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8857789751790294) > weight (0.25) 2021-02-20T08:55:20.179 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5048011617506936) > weight (0.25) 2021-02-20T08:55:20.179 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9880677728100051) > weight (0.25) 2021-02-20T08:55:20.179 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T08:55:20.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T08:55:20.181 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T08:55:20.182 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T08:55:20.177+0000 7f15e4b40700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 35341) UID: 1000 2021-02-20T08:55:20.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:55:24.181 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T08:55:24.182 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:55:24.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:55:24.183 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T08:55:24.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T08:55:24.184 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T08:55:24.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T08:55:24.186 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T08:55:33.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3103146436084093) > weight (0.25) 2021-02-20T08:55:33.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.30670032297761907) > weight (0.25) 2021-02-20T08:55:33.375 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T08:55:33.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T08:55:33.376 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T08:55:33.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T08:55:33.373+0000 7f97b0a8c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35628) UID: 1000 2021-02-20T08:55:33.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:55:42.377 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T08:55:42.378 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:55:42.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:55:42.378 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T08:55:42.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T08:55:42.379 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T08:55:42.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T08:55:42.382 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T08:57:11.188 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8248370329462843) > weight (0.25) 2021-02-20T08:57:11.198 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T08:57:11.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T08:57:11.199 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:57:11.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T08:57:11.199+0000 7f2af2e2a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35729) UID: 1000 2021-02-20T08:57:11.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:57:13.383 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T08:57:13.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T08:57:13.384 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:57:13.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T08:57:13.382+0000 7fc0c1ce0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 34005) UID: 1000 2021-02-20T08:57:13.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:57:15.385 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T08:57:15.385 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:57:15.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:57:15.386 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T08:57:15.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T08:57:15.387 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T08:57:15.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T08:57:15.390 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T08:57:18.200 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T08:57:18.200 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:57:18.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:57:18.201 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T08:57:18.201 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T08:57:18.202 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T08:57:18.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T08:57:18.205 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T08:58:44.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26663494531088683) > weight (0.25) 2021-02-20T08:58:44.212 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9060228320422227) > weight (0.25) 2021-02-20T08:58:44.212 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T08:58:44.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T08:58:44.213 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T08:58:44.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T08:58:44.209+0000 7f11b971a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35435) UID: 1000 2021-02-20T08:58:44.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:58:46.214 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T08:58:46.214 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:58:46.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:58:46.215 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T08:58:46.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T08:58:46.216 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T08:58:46.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T08:58:46.219 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T08:59:08.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8739891680028998) > weight (0.25) 2021-02-20T08:59:08.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9007123447994179) > weight (0.25) 2021-02-20T08:59:08.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8628589969032251) > weight (0.25) 2021-02-20T08:59:08.397 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T08:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T08:59:08.398 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T08:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T08:59:08.392+0000 7f891bd5c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 35820) UID: 1000 2021-02-20T08:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T08:59:15.399 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T08:59:15.399 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T08:59:15.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T08:59:15.400 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T08:59:15.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T08:59:15.401 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T08:59:15.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T08:59:15.404 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T09:00:35.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6106972708364302) > weight (0.25) 2021-02-20T09:00:35.221 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:00:35.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:00:35.223 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:00:35.223 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:00:35.223 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:00:35.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:00:35.218+0000 7fc9a3004700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36298) UID: 1000 2021-02-20T09:00:35.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:00:39.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48064638280400407) > weight (0.25) 2021-02-20T09:00:39.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8645920170391979) > weight (0.25) 2021-02-20T09:00:39.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27956869001281703) > weight (0.25) 2021-02-20T09:00:39.406 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:00:39.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:00:39.407 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:00:39.407 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:00:39.408 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:00:39.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T09:00:39.406+0000 7f403bb1c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 36491) UID: 1000 2021-02-20T09:00:39.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:00:41.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:00:41.326 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:00:41.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:00:41.327 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:00:41.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:00:41.333 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T09:00:45.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:00:45.524 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:00:45.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:00:45.525 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:00:45.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:00:45.533 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T09:01:10.365 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:01:10.361+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d949810 2021-02-20T09:01:10.366 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:01:10.361+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b991e0 2021-02-20T09:02:05.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5473056571451491) > weight (0.25) 2021-02-20T09:02:05.541 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:02:05.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:02:05.542 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:02:05.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:02:05.536+0000 7ff01f804700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35526) UID: 1000 2021-02-20T09:02:05.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:02:07.544 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:02:07.551 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:02:07.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:02:07.552 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:02:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:02:07.553 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:02:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:02:07.557 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T09:02:33.335 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7414825166772288) > weight (0.25) 2021-02-20T09:02:33.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7990255223510138) > weight (0.25) 2021-02-20T09:02:33.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3240000857645168) > weight (0.25) 2021-02-20T09:02:33.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8989928709509878) > weight (0.25) 2021-02-20T09:02:33.345 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T09:03:54.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6416533271811223) > weight (0.25) 2021-02-20T09:03:54.638 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9327709784938134) > weight (0.25) 2021-02-20T09:03:54.639 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:03:54.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:03:54.641 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:03:54.642 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:03:54.644 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:03:54.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T09:03:54.638+0000 7f1cb461d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 36108) UID: 1000 2021-02-20T09:03:55.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:03:56.202 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:03:56.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d947790 2021-02-20T09:03:56.203 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:03:56.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d949260 2021-02-20T09:03:56.203 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:03:56.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac846e83c0 2021-02-20T09:03:56.203 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:03:56.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133b380 2021-02-20T09:03:56.203 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:03:56.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e21450 2021-02-20T09:04:00.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:04:00.747 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:04:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:04:00.748 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:04:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:04:00.751 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T09:04:21.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5728365174247368) > weight (0.25) 2021-02-20T09:04:21.354 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8938911619557828) > weight (0.25) 2021-02-20T09:04:21.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.89938799633594) > weight (0.25) 2021-02-20T09:04:21.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7079922808257458) > weight (0.25) 2021-02-20T09:04:21.355 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:05:56.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6155683492597974) > weight (0.25) 2021-02-20T09:05:56.754 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:05:56.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:05:56.756 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:05:56.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:05:56.751+0000 7fb73713c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36781) UID: 1000 2021-02-20T09:05:57.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:05:59.757 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:05:59.764 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:05:59.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:05:59.765 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:05:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:05:59.765 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:05:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:05:59.769 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T09:06:14.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6291692287590218) > weight (0.25) 2021-02-20T09:06:14.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9286691327683484) > weight (0.25) 2021-02-20T09:06:14.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2520463969598903) > weight (0.25) 2021-02-20T09:06:14.367 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3576339227574422) > weight (0.25) 2021-02-20T09:06:14.368 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T09:07:32.368 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6434430063156859) > weight (0.25) 2021-02-20T09:07:32.383 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4433703482997913) > weight (0.25) 2021-02-20T09:07:32.383 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:07:32.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:07:32.385 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:07:32.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:07:32.381+0000 7efcb96f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 36400) UID: 1000 2021-02-20T09:07:32.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:07:36.769 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:07:36.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:07:36.778 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:07:36.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:07:36.773+0000 7f5862258700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36207) UID: 1000 2021-02-20T09:07:37.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:07:37.387 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:07:37.387 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:07:37.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:07:37.389 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:07:37.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:07:37.390 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:07:37.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:07:37.395 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T09:07:42.779 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:07:42.784 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:07:42.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:07:42.786 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:07:42.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:07:42.787 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:07:42.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:07:42.797 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T09:08:44.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4199751908534213) > weight (0.25) 2021-02-20T09:08:44.806 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48609651888984395) > weight (0.25) 2021-02-20T09:08:44.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7553177577279385) > weight (0.25) 2021-02-20T09:08:44.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7322467335750356) > weight (0.25) 2021-02-20T09:08:44.807 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T09:08:49.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9534614653388923) > weight (0.25) 2021-02-20T09:08:49.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5321150002578915) > weight (0.25) 2021-02-20T09:08:49.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8939945061174293) > weight (0.25) 2021-02-20T09:08:49.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8257910746544135) > weight (0.25) 2021-02-20T09:08:49.399 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T09:09:47.809 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:09:47.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:09:47.811 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:09:47.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:09:47.810+0000 7f2be9474700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37176) UID: 1000 2021-02-20T09:09:48.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:09:51.812 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:09:51.812 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:09:51.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:09:51.813 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:09:51.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:09:51.814 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:09:51.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:09:51.817 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T09:10:46.401 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8960536725952977) > weight (0.25) 2021-02-20T09:10:46.402 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:10:46.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:10:46.403 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:10:46.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:10:46.401+0000 7f3aed3c7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 36591) UID: 1000 2021-02-20T09:10:46.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:10:54.404 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:10:54.413 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:10:54.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:10:54.414 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:10:54.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:10:54.416 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:10:54.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:10:54.419 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T09:11:31.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9650321911296932) > weight (0.25) 2021-02-20T09:11:31.830 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:11:31.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:11:31.831 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:11:31.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:11:31.828+0000 7f70c690a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 36986) UID: 1000 2021-02-20T09:11:32.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:11:33.833 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:11:33.833 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:11:33.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:11:33.834 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:11:33.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:11:33.834 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:11:33.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:11:33.837 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T09:11:40.369 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:11:40.366+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d949810 2021-02-20T09:11:40.370 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:11:40.366+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b991e0 2021-02-20T09:12:07.420 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:12:07.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:12:07.423 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:12:07.423 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:12:07.424 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:12:07.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:12:07.419+0000 7ffa41949700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35919) UID: 1000 2021-02-20T09:12:07.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:12:13.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:12:13.526 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:12:13.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:12:13.527 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:12:13.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:12:13.530 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T09:13:01.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28993708084534553) > weight (0.25) 2021-02-20T09:13:01.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9301783218993203) > weight (0.25) 2021-02-20T09:13:01.849 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:13:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:13:01.850 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:13:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T09:13:01.846+0000 7f6cb8d46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 36883) UID: 1000 2021-02-20T09:13:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:13:09.852 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:13:09.852 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:13:09.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:13:09.853 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:13:09.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:13:09.854 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:13:09.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:13:09.857 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T09:13:20.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9708557261419519) > weight (0.25) 2021-02-20T09:13:20.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6079131913193572) > weight (0.25) 2021-02-20T09:13:20.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5422174978775923) > weight (0.25) 2021-02-20T09:13:20.540 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:13:20.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:13:20.541 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:13:20.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T09:13:20.538+0000 7fb82729a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 36017) UID: 1000 2021-02-20T09:13:20.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:13:25.542 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:13:25.550 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:13:25.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:13:25.551 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:13:25.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:13:25.551 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:13:25.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:13:25.554 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T09:14:25.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5796809595037513) > weight (0.25) 2021-02-20T09:14:25.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5279499242285789) > weight (0.25) 2021-02-20T09:14:25.567 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:14:25.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:14:25.568 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T09:14:25.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:25.564+0000 7ffb60b83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 37085) UID: 1000 2021-02-20T09:14:25.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:14:34.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:14:34.576 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:14:34.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:14:34.577 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:14:34.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:14:34.578 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:14:34.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:14:34.581 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T09:14:48.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6311367258433322) > weight (0.25) 2021-02-20T09:14:48.866 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:14:48.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:14:48.867 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:14:48.868 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:14:48.868 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:14:48.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:14:48.864+0000 7f470a855700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 37470) UID: 1000 2021-02-20T09:14:49.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:14:54.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:14:54.976 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:14:54.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:14:54.978 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:14:54.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:14:54.988 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T09:14:59.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff605900 [3/3ca6eed3-b4cc-42a3-8669-fd637c7a8d69] start: already running 2021-02-20T09:14:59.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff605b80 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T09:14:59.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff605e00 [3/4d947326-1304-4f75-9cb5-1cc9895ee2c8] start: already running 2021-02-20T09:14:59.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff606080 [3/5d895514-421a-41d1-8f87-96dea12501c8] start: already running 2021-02-20T09:14:59.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff606300 [3/7cc83948-bf26-4ed7-a142-2db0f2786e6e] start: already running 2021-02-20T09:14:59.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:14:59.004+0000 7f1890e30700 -1 rbd::mirror::ImageReplayer: 0x5561ff606580 [3/b7d39708-c03f-469a-a6fa-228e371ae7a3] start: already running 2021-02-20T09:15:37.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5373191781060691) > weight (0.25) 2021-02-20T09:15:37.589 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8660859699660732) > weight (0.25) 2021-02-20T09:15:37.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9558351521673103) > weight (0.25) 2021-02-20T09:15:37.590 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:15:37.591 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:15:37.592 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:15:37.592 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:15:37.593 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:15:37.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T09:15:37.591+0000 7f82835ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 37760) UID: 1000 2021-02-20T09:15:37.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:15:43.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:15:43.703 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:15:43.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:15:43.704 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:15:43.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:15:43.708 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:16:18.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8592830380193703) > weight (0.25) 2021-02-20T09:16:18.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.974207827565808) > weight (0.25) 2021-02-20T09:16:18.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3211579020621508) > weight (0.25) 2021-02-20T09:16:18.990 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:16:18.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:16:18.991 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:16:18.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T09:16:18.990+0000 7fc6862d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 36682) UID: 1000 2021-02-20T09:16:19.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:16:22.992 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:16:23.000 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:16:23.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:16:23.001 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:16:23.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:16:23.002 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:16:23.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:16:23.009 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T09:17:36.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6010653459252515) > weight (0.25) 2021-02-20T09:17:36.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9514277594115481) > weight (0.25) 2021-02-20T09:17:36.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.49505223077329785) > weight (0.25) 2021-02-20T09:17:36.746 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:17:36.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:17:36.748 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:17:36.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T09:17:36.744+0000 7efd469b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 38021) UID: 1000 2021-02-20T09:17:36.948 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:17:44.751 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:17:44.751 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:17:44.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:17:44.752 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:17:44.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:17:44.753 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:17:44.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:17:44.761 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T09:18:04.842 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:18:04.837+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750a340 2021-02-20T09:18:09.284 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:18:09.277+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133ba00 2021-02-20T09:18:13.011 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.687353831933372) > weight (0.25) 2021-02-20T09:18:13.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6704446946907062) > weight (0.25) 2021-02-20T09:18:13.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30724712292716916) > weight (0.25) 2021-02-20T09:18:13.018 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:18:13.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:18:13.019 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:18:13.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T09:18:13.011+0000 7fb6df9fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 38112) UID: 1000 2021-02-20T09:18:13.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:18:21.019 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:18:21.020 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:18:21.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:18:21.021 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:18:21.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:18:21.022 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:18:21.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:18:21.025 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T09:18:54.762 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:18:54.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:18:54.770 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:18:54.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:18:54.766+0000 7fe37ca39700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37569) UID: 1000 2021-02-20T09:18:55.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:18:58.771 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:18:58.772 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:18:58.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:18:58.773 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:18:58.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:18:58.773 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:18:58.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:18:58.776 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:19:48.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9933721053977812) > weight (0.25) 2021-02-20T09:19:48.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8644552928646256) > weight (0.25) 2021-02-20T09:19:48.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.377324311001745) > weight (0.25) 2021-02-20T09:19:48.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.26075747637727464) > weight (0.25) 2021-02-20T09:19:48.035 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T09:20:51.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6394293422053267) > weight (0.25) 2021-02-20T09:20:51.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.35758559988934) > weight (0.25) 2021-02-20T09:20:51.785 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:20:51.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:20:51.786 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:20:51.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:20:51.788+0000 7f1891631700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 37853) UID: 1000 2021-02-20T09:20:52.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:20:53.787 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:20:53.788 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:20:53.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:20:53.788 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:20:53.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:20:53.789 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:20:53.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:20:53.792 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T09:21:30.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4466766549116721) > weight (0.25) 2021-02-20T09:21:30.038 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:21:30.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:21:30.039 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:21:30.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:21:30.039+0000 7f93b3127700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 37944) UID: 1000 2021-02-20T09:21:30.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:21:37.040 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:21:37.040 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:21:37.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:21:37.041 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:21:37.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:21:37.042 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:21:37.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:21:37.045 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T09:22:37.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7702632292661864) > weight (0.25) 2021-02-20T09:22:37.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5958716606794352) > weight (0.25) 2021-02-20T09:22:37.049 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:22:37.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:22:37.050 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:22:37.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T09:22:37.046+0000 7f4f7d980700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 37661) UID: 1000 2021-02-20T09:22:37.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:22:42.050 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:22:42.051 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:22:42.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:22:42.052 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:22:42.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:22:42.052 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:22:42.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:22:42.056 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T09:22:47.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5265365275813051) > weight (0.25) 2021-02-20T09:22:47.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9221647077876503) > weight (0.25) 2021-02-20T09:22:47.795 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:22:47.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:22:47.795 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:22:47.796 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:22:47.796 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:22:47.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:22:47.793+0000 7f22e4188700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38527) UID: 1000 2021-02-20T09:22:48.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:22:53.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:22:53.899 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:22:53.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:22:53.900 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:22:53.903 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:22:53.903 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T09:23:57.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7198784853930414) > weight (0.25) 2021-02-20T09:23:57.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8620344833583946) > weight (0.25) 2021-02-20T09:23:57.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25620399703122143) > weight (0.25) 2021-02-20T09:23:57.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.30875705602871506) > weight (0.25) 2021-02-20T09:23:57.906 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T09:24:19.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6415848891244712) > weight (0.25) 2021-02-20T09:24:19.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5192650081383572) > weight (0.25) 2021-02-20T09:24:19.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8562304903310182) > weight (0.25) 2021-02-20T09:24:19.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.525989093625183) > weight (0.25) 2021-02-20T09:24:19.066 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T09:25:19.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3821017019191236) > weight (0.25) 2021-02-20T09:25:19.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7292924981725717) > weight (0.25) 2021-02-20T09:25:19.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5599903390822496) > weight (0.25) 2021-02-20T09:25:19.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5188257310995685) > weight (0.25) 2021-02-20T09:25:19.920 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T09:25:28.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4916598083366879) > weight (0.25) 2021-02-20T09:25:28.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8654143003071775) > weight (0.25) 2021-02-20T09:25:28.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43350294798402866) > weight (0.25) 2021-02-20T09:25:28.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8077368217031378) > weight (0.25) 2021-02-20T09:25:28.074 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T09:26:58.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6530215193274004) > weight (0.25) 2021-02-20T09:26:58.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6215899964753662) > weight (0.25) 2021-02-20T09:26:58.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41180556976169735) > weight (0.25) 2021-02-20T09:26:58.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7393642698368462) > weight (0.25) 2021-02-20T09:26:58.106 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:27:18.920 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9213670048517787) > weight (0.25) 2021-02-20T09:27:18.928 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33203155330212664) > weight (0.25) 2021-02-20T09:27:18.929 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:27:18.930 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:27:18.930 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:27:18.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:27:18.928+0000 7f8049390700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38816) UID: 1000 2021-02-20T09:27:19.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:27:25.931 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:27:25.940 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:27:25.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:27:25.941 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:27:25.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:27:25.942 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:27:25.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:27:25.945 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T09:28:46.946 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29746717665063815) > weight (0.25) 2021-02-20T09:28:46.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.535986688989232) > weight (0.25) 2021-02-20T09:28:46.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2510598705082726) > weight (0.25) 2021-02-20T09:28:46.954 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T09:28:46.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T09:28:46.956 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:28:46.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T09:28:46.957 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:28:46.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T09:28:46.954+0000 7ff179639700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 38214) UID: 1000 2021-02-20T09:28:47.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:28:51.108 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:28:51.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:28:51.115 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:28:51.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:28:51.110+0000 7fd36ad08700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37279) UID: 1000 2021-02-20T09:28:51.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:28:53.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:28:53.060 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T09:28:53.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T09:28:53.061 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T09:28:53.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T09:28:53.065 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T09:28:53.115 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:28:53.116 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:28:53.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:28:53.116 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:28:53.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:28:53.117 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:28:53.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:28:53.120 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T09:30:34.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3762315428234402) > weight (0.25) 2021-02-20T09:30:34.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8148189840993404) > weight (0.25) 2021-02-20T09:30:34.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6667190476603252) > weight (0.25) 2021-02-20T09:30:34.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5572708775260554) > weight (0.25) 2021-02-20T09:30:34.074 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T09:30:52.121 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:30:52.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:30:52.123 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:30:52.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:30:52.119+0000 7f500da5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39008) UID: 1000 2021-02-20T09:30:52.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:30:56.123 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:30:56.124 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:30:56.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:30:56.126 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:30:56.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:30:56.127 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:30:56.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:30:56.131 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T09:31:42.273 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T09:31:42.270+0000 7f803c4bb700 -1 reset not still connected to 0x55acb88ec4e0 2021-02-20T09:31:42.327 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T09:31:42.274+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8e285b0 2021-02-20T09:32:11.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7130256800024052) > weight (0.25) 2021-02-20T09:32:11.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9480155086505138) > weight (0.25) 2021-02-20T09:32:11.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5603632323839634) > weight (0.25) 2021-02-20T09:32:11.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.30224923479556376) > weight (0.25) 2021-02-20T09:32:11.078 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T09:32:36.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7323483929308401) > weight (0.25) 2021-02-20T09:32:36.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39196971570864303) > weight (0.25) 2021-02-20T09:32:36.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.779246011978875) > weight (0.25) 2021-02-20T09:32:36.138 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:32:36.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:32:36.139 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:32:36.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T09:32:36.133+0000 7f1febd24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 38306) UID: 1000 2021-02-20T09:32:36.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:32:44.140 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:32:44.145 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:32:44.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:32:44.146 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:32:44.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:32:44.147 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:32:44.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:32:44.153 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T09:33:42.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5018712037515808) > weight (0.25) 2021-02-20T09:33:42.089 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:33:42.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:33:42.090 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:33:42.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:33:42.087+0000 7fe4fdb43700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 37379) UID: 1000 2021-02-20T09:33:42.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:33:43.091 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:33:43.091 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:33:43.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:33:43.092 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:33:43.092 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:33:43.093 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:33:43.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:33:43.096 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T09:34:20.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39745013124221473) > weight (0.25) 2021-02-20T09:34:20.162 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:34:20.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:34:20.163 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:34:20.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:34:20.158+0000 7fd446149700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38618) UID: 1000 2021-02-20T09:34:20.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:34:24.165 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:34:24.172 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:34:24.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:34:24.173 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:34:24.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:34:24.173 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:34:24.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:34:24.176 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T09:35:10.098 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4417135879872559) > weight (0.25) 2021-02-20T09:35:10.105 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:35:10.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:35:10.107 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:35:10.107 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:35:10.107 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:35:10.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:35:10.101+0000 7f12bf746700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39399) UID: 1000 2021-02-20T09:35:10.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:35:16.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:35:16.209 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:35:16.210 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:35:16.210 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:35:16.215 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:35:16.215 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T09:35:56.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.774011755610784) > weight (0.25) 2021-02-20T09:35:56.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:35:56.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:35:56.189 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:35:56.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:35:56.184+0000 7f9f6bb3d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 39490) UID: 1000 2021-02-20T09:35:56.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:36:02.191 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:36:02.192 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:36:02.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:36:02.193 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:36:02.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:36:02.194 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:36:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:36:02.198 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T09:37:07.200 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:37:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:37:07.202 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:37:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:37:07.199+0000 7f950cacc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39199) UID: 1000 2021-02-20T09:37:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:37:10.202 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:37:10.203 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:37:10.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:37:10.204 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:37:10.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:37:10.205 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:37:10.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:37:10.208 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T09:37:15.216 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7884240882966214) > weight (0.25) 2021-02-20T09:37:15.224 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7714790770338373) > weight (0.25) 2021-02-20T09:37:15.224 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:37:15.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:37:15.225 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:37:15.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:37:15.223+0000 7f2a3880a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 38914) UID: 1000 2021-02-20T09:37:15.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:37:21.226 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:37:21.227 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:37:21.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:37:21.227 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:37:21.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:37:21.228 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:37:21.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:37:21.231 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T09:37:23.302 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:37:23.298+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b99a00 2021-02-20T09:37:23.303 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:37:23.298+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750d260 2021-02-20T09:38:15.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2962570129238349) > weight (0.25) 2021-02-20T09:38:15.209 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:38:15.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:38:15.210 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:38:15.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:38:15.205+0000 7fc51d3c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 39683) UID: 1000 2021-02-20T09:38:15.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:38:18.212 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:38:18.212 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:38:18.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:38:18.213 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:38:18.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:38:18.214 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:38:18.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:38:18.217 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T09:38:22.919 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:38:22.912+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133c7d0 2021-02-20T09:39:16.232 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.975627456269943) > weight (0.25) 2021-02-20T09:39:16.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.31171627281237146) > weight (0.25) 2021-02-20T09:39:16.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9227171619736091) > weight (0.25) 2021-02-20T09:39:16.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8683758267880449) > weight (0.25) 2021-02-20T09:39:16.241 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T09:40:02.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.977408669240068) > weight (0.25) 2021-02-20T09:40:02.219 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:40:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:40:02.220 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:40:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:40:02.219+0000 7f2dca958700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 39975) UID: 1000 2021-02-20T09:40:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:40:03.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:40:03.222 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:40:03.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:40:03.222 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:40:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:40:03.223 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:40:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:40:03.226 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T09:40:38.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43047272098860523) > weight (0.25) 2021-02-20T09:40:38.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7425918109922416) > weight (0.25) 2021-02-20T09:40:38.244 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:40:38.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:40:38.246 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T09:40:38.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:40:38.242+0000 7f04601d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 39884) UID: 1000 2021-02-20T09:40:38.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:40:40.247 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:40:40.256 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:40:40.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:40:40.256 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:40:40.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:40:40.257 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:40:40.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:40:40.261 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T09:41:53.227 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4472580362184617) > weight (0.25) 2021-02-20T09:41:53.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6144783728437281) > weight (0.25) 2021-02-20T09:41:53.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7450991108334074) > weight (0.25) 2021-02-20T09:41:53.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3464788318012316) > weight (0.25) 2021-02-20T09:41:53.236 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T09:42:07.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39040487949394653) > weight (0.25) 2021-02-20T09:42:07.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49557131426905343) > weight (0.25) 2021-02-20T09:42:07.271 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T09:42:07.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T09:42:07.273 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:42:07.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T09:42:07.268+0000 7f8f637f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40175) UID: 1000 2021-02-20T09:42:07.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:42:10.273 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T09:42:10.282 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:42:10.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:42:10.283 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T09:42:10.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T09:42:10.284 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T09:42:10.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T09:42:10.287 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T09:43:41.237 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5191433559555991) > weight (0.25) 2021-02-20T09:43:41.238 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7173463735549632) > weight (0.25) 2021-02-20T09:43:41.238 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:43:41.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:43:41.240 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:43:41.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T09:43:41.238+0000 7f5a9c3a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 38717) UID: 1000 2021-02-20T09:43:41.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:43:48.241 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:43:48.241 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:43:48.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:43:48.242 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:43:48.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:43:48.243 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:43:48.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:43:48.246 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T09:43:50.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9205893163607302) > weight (0.25) 2021-02-20T09:43:50.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5466032966671522) > weight (0.25) 2021-02-20T09:43:50.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3158946142775717) > weight (0.25) 2021-02-20T09:43:50.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3692946692009623) > weight (0.25) 2021-02-20T09:43:50.289 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T09:45:34.290 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:45:34.291 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:45:34.291 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T09:45:34.292 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:45:34.284+0000 7efc49be9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38406) UID: 1000 2021-02-20T09:45:34.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:45:35.247 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:45:35.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:45:35.248 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T09:45:35.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:45:35.244+0000 7f78d9bf0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39785) UID: 1000 2021-02-20T09:45:35.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:45:37.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:45:37.293 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:45:37.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:45:37.294 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:45:37.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:45:37.295 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:45:37.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:45:37.298 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T09:45:42.248 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:45:42.249 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:45:42.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:45:42.251 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:45:42.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:45:42.251 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:45:42.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:45:42.255 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T09:46:50.257 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T09:46:50.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T09:46:50.268 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T09:46:50.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:46:50.262+0000 7fd0e716d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40553) UID: 1000 2021-02-20T09:46:50.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:46:51.269 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T09:46:51.270 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:46:51.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:46:51.271 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T09:46:51.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T09:46:51.271 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T09:46:51.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T09:46:51.275 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T09:47:12.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6080020843767362) > weight (0.25) 2021-02-20T09:47:12.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8783056856064877) > weight (0.25) 2021-02-20T09:47:12.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4633031909405201) > weight (0.25) 2021-02-20T09:47:12.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2913830661941913) > weight (0.25) 2021-02-20T09:47:12.309 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T09:48:28.310 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:48:28.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:48:28.312 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T09:48:28.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:48:28.308+0000 7ffba9cd9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40462) UID: 1000 2021-02-20T09:48:28.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:48:33.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6818911351097116) > weight (0.25) 2021-02-20T09:48:33.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.740006626209247) > weight (0.25) 2021-02-20T09:48:33.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3004369685764623) > weight (0.25) 2021-02-20T09:48:33.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5350475763454253) > weight (0.25) 2021-02-20T09:48:33.277 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T09:48:34.313 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:48:34.767 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:48:34.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:48:34.768 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:48:34.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:48:34.769 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:48:34.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:48:34.776 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T09:49:50.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7628805163625688) > weight (0.25) 2021-02-20T09:49:50.281 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:49:50.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:49:50.282 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T09:49:50.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:49:50.278+0000 7fe208acf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40075) UID: 1000 2021-02-20T09:49:50.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:49:59.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:49:59.284 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:49:59.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:49:59.285 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:49:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:49:59.286 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:49:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:49:59.289 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T09:50:19.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.375436885529161) > weight (0.25) 2021-02-20T09:50:19.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8474548340719738) > weight (0.25) 2021-02-20T09:50:19.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7615728955000716) > weight (0.25) 2021-02-20T09:50:19.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8767405190991444) > weight (0.25) 2021-02-20T09:50:19.789 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T09:51:11.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6975189800087961) > weight (0.25) 2021-02-20T09:51:11.299 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:51:11.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:51:11.301 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:51:11.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:51:11.297+0000 7f64fd6af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40846) UID: 1000 2021-02-20T09:51:11.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:51:15.301 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:51:15.312 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:51:15.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:51:15.313 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:51:15.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:51:15.314 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:51:15.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:51:15.317 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T09:52:09.789 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:52:09.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:52:09.791 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T09:52:09.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:52:09.787+0000 7fdc78c92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 40754) UID: 1000 2021-02-20T09:52:10.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:52:17.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:52:17.804 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:52:17.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:52:17.805 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:52:17.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:52:17.805 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:52:17.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:52:17.809 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T09:52:40.319 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3329701184718037) > weight (0.25) 2021-02-20T09:52:40.327 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.688973851330184) > weight (0.25) 2021-02-20T09:52:40.328 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6813876556224363) > weight (0.25) 2021-02-20T09:52:40.328 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T09:52:40.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T09:52:40.329 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:52:40.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T09:52:40.327+0000 7fb1747e8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 39300) UID: 1000 2021-02-20T09:52:40.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:52:41.526 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T09:52:41.523+0000 7f803c4bb700 -1 reset not still connected to 0x55acb55358e0 2021-02-20T09:52:41.527 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T09:52:41.523+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8e2b330 2021-02-20T09:52:44.330 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T09:52:44.330 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:52:44.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:52:44.331 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T09:52:44.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T09:52:44.332 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T09:52:44.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T09:52:44.335 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T09:53:10.539 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:53:10.535+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f0f2e50 2021-02-20T09:53:10.540 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:53:10.535+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750ad00 2021-02-20T09:53:10.540 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T09:53:10.535+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750b930 2021-02-20T09:53:46.810 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.802834019408423) > weight (0.25) 2021-02-20T09:53:46.820 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:53:46.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:53:46.822 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T09:53:46.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:53:46.817+0000 7f84688e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39592) UID: 1000 2021-02-20T09:53:47.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:53:51.824 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:53:51.830 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:53:51.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:53:51.831 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:53:51.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:53:51.832 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:53:51.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:53:51.835 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T09:54:19.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5669921397628334) > weight (0.25) 2021-02-20T09:54:19.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6066251641848031) > weight (0.25) 2021-02-20T09:54:19.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4008924523585392) > weight (0.25) 2021-02-20T09:54:19.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.662232367545495) > weight (0.25) 2021-02-20T09:54:19.385 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T09:55:16.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4296070814317613) > weight (0.25) 2021-02-20T09:55:16.902 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T09:55:16.903 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T09:55:16.903 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T09:55:16.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T09:55:16.899+0000 7fc4631e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41238) UID: 1000 2021-02-20T09:55:17.305 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:55:25.904 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T09:55:25.912 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:55:25.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:55:25.913 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T09:55:25.913 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T09:55:25.915 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T09:55:25.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T09:55:25.918 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T09:56:10.386 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.697830900724423) > weight (0.25) 2021-02-20T09:56:10.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9164383971818397) > weight (0.25) 2021-02-20T09:56:10.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.44723388743399095) > weight (0.25) 2021-02-20T09:56:10.388 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5060066484544059) > weight (0.25) 2021-02-20T09:56:10.388 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T09:57:18.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46936551010851724) > weight (0.25) 2021-02-20T09:57:18.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8269860566623192) > weight (0.25) 2021-02-20T09:57:18.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7794950893083018) > weight (0.25) 2021-02-20T09:57:18.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5295758857546226) > weight (0.25) 2021-02-20T09:57:18.959 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T09:57:49.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5766341570978855) > weight (0.25) 2021-02-20T09:57:49.414 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T09:57:49.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T09:57:49.416 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:57:49.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T09:57:49.412+0000 7fca7041e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 40946) UID: 1000 2021-02-20T09:57:49.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:57:53.417 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T09:57:54.534 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:57:54.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:57:54.546 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T09:57:54.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T09:57:54.547 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T09:57:54.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T09:57:54.553 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T09:58:21.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T09:58:21.079+0000 7f73b5217700 -1 rbd::mirror::ImageReplayer: 0x556565bd7700 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T09:59:14.960 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T09:59:15.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T09:59:15.004 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T09:59:15.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T09:59:15.002+0000 7f52a2e69700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41045) UID: 1000 2021-02-20T09:59:15.406 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:59:19.005 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T09:59:19.010 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:59:19.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:59:19.011 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T09:59:19.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T09:59:19.012 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T09:59:19.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T09:59:19.015 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T09:59:36.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3723066474234227) > weight (0.25) 2021-02-20T09:59:36.561 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9997132309324559) > weight (0.25) 2021-02-20T09:59:36.561 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T09:59:36.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T09:59:36.562 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T09:59:36.563 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T09:59:36.563 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T09:59:36.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T09:59:36.562+0000 7f6de9763700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 40360) UID: 1000 2021-02-20T09:59:36.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T09:59:42.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T09:59:42.672 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T09:59:42.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T09:59:42.673 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T09:59:42.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T09:59:42.676 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T10:00:23.317 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:00:23.313+0000 7f803c4bb700 -1 reset not still connected to 0x55acbadbe2f0 2021-02-20T10:00:23.319 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:00:23.313+0000 7f803c4bb700 -1 reset not still connected to 0x55acea49ed00 2021-02-20T10:00:29.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46738806009885514) > weight (0.25) 2021-02-20T10:00:29.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5481921102578774) > weight (0.25) 2021-02-20T10:00:29.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2943776871882732) > weight (0.25) 2021-02-20T10:00:29.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6978553445276295) > weight (0.25) 2021-02-20T10:00:29.030 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T10:00:59.678 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43898566253459514) > weight (0.25) 2021-02-20T10:00:59.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7829087555947497) > weight (0.25) 2021-02-20T10:00:59.686 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3511154777623323) > weight (0.25) 2021-02-20T10:00:59.687 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T10:00:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T10:00:59.689 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:00:59.689 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T10:00:59.690 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:00:59.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T10:00:59.688+0000 7f4001d54700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41136) UID: 1000 2021-02-20T10:01:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:01:05.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:01:05.800 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T10:01:05.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T10:01:05.802 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T10:01:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T10:01:05.808 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T10:02:05.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2851723973648528) > weight (0.25) 2021-02-20T10:02:05.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6348962065313732) > weight (0.25) 2021-02-20T10:02:05.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.944329995416908) > weight (0.25) 2021-02-20T10:02:05.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5164631210578114) > weight (0.25) 2021-02-20T10:02:05.076 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T10:02:18.809 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5268528002382038) > weight (0.25) 2021-02-20T10:02:18.810 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:02:18.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:02:18.812 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:02:18.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:02:18.810+0000 7fc8cd24b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41423) UID: 1000 2021-02-20T10:02:19.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:02:20.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:02:20.821 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:02:20.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:02:20.822 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:02:20.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:02:20.824 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:02:20.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:02:20.836 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T10:03:08.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5841136957494693) > weight (0.25) 2021-02-20T10:03:08.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8169040559302865) > weight (0.25) 2021-02-20T10:03:08.088 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:03:08.089 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:03:08.090 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:03:08.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:03:08.093+0000 7f2086557700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40268) UID: 1000 2021-02-20T10:03:08.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:03:11.091 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:03:11.098 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:03:11.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:03:11.099 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:03:11.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:03:11.099 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:03:11.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:03:11.103 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T10:03:52.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2591646928581852) > weight (0.25) 2021-02-20T10:03:52.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46739549415779513) > weight (0.25) 2021-02-20T10:03:52.847 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:03:52.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:03:52.849 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:03:52.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T10:03:52.844+0000 7f21f508e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 41617) UID: 1000 2021-02-20T10:03:53.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:03:53.305 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:03:53.300+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b99a00 2021-02-20T10:03:58.851 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:03:58.864 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:03:58.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:03:58.865 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:03:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:03:58.866 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:03:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:03:58.870 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T10:04:24.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6077599663748517) > weight (0.25) 2021-02-20T10:04:24.145 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8054659228015147) > weight (0.25) 2021-02-20T10:04:24.145 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:04:24.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:04:24.147 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:04:24.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:04:24.143+0000 7f77b01b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41915) UID: 1000 2021-02-20T10:04:24.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:04:26.147 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:04:26.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:04:26.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:04:26.153 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:04:26.154 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:04:26.154 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:04:26.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:04:26.157 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:05:40.872 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:05:40.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:05:40.883 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:05:40.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:05:40.882+0000 7f73b5a18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40652) UID: 1000 2021-02-20T10:05:41.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:05:49.884 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:05:49.892 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:05:49.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:05:49.894 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:05:49.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:05:49.896 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:05:49.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:05:49.902 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T10:06:17.157 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:06:17.191 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:06:17.191 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:06:17.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:06:17.189+0000 7fc9c52e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41526) UID: 1000 2021-02-20T10:06:17.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:06:25.192 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:06:25.192 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:06:25.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:06:25.193 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:06:25.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:06:25.194 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:06:25.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:06:25.197 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T10:07:16.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.718238285055166) > weight (0.25) 2021-02-20T10:07:16.923 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:07:16.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:07:16.924 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:07:16.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:07:16.924+0000 7fc06bfca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41816) UID: 1000 2021-02-20T10:07:17.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:07:24.926 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:07:24.932 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:07:24.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:07:24.933 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:07:24.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:07:24.934 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:07:24.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:07:24.940 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T10:08:18.198 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5575568928471551) > weight (0.25) 2021-02-20T10:08:18.445 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:08:18.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:08:18.449 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:08:18.450 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:08:18.446+0000 7fe652fa3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41332) UID: 1000 2021-02-20T10:08:18.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:08:19.450 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:08:19.455 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:08:19.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:08:19.456 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:08:19.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:08:19.457 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:08:19.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:08:19.461 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T10:09:08.941 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43518277426491303) > weight (0.25) 2021-02-20T10:09:08.950 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:09:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:09:08.951 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:09:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:09:08.953+0000 7f1f06c4e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42393) UID: 1000 2021-02-20T10:09:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:09:14.953 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:09:14.959 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:09:14.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:09:14.961 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:09:14.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:09:14.963 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:09:14.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:09:14.967 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T10:09:39.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5565926093204177) > weight (0.25) 2021-02-20T10:09:39.476 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:09:39.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:09:39.477 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:09:39.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:09:39.473+0000 7f8573846700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 42492) UID: 1000 2021-02-20T10:09:39.878 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:09:41.477 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:09:41.481 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:09:41.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:09:41.482 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:09:41.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:09:41.483 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:09:41.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:09:41.487 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T10:10:26.399 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:10:26.395+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b9b740 2021-02-20T10:10:38.970 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:10:38.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:10:38.972 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:10:38.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:10:38.967+0000 7f9411fa2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42203) UID: 1000 2021-02-20T10:10:39.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:10:40.972 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:10:40.983 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:10:40.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:10:40.984 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:10:40.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:10:40.984 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:10:40.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:10:40.989 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T10:10:53.488 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:10:53.497 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:10:53.497 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:10:53.498 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:10:53.491+0000 7f0139c03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42302) UID: 1000 2021-02-20T10:10:53.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:10:56.499 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:10:56.510 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:10:56.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:10:56.512 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:10:56.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:10:56.512 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:10:56.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:10:56.516 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T10:12:00.517 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5805303018380751) > weight (0.25) 2021-02-20T10:12:00.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8886436066230895) > weight (0.25) 2021-02-20T10:12:00.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8035249416860671) > weight (0.25) 2021-02-20T10:12:00.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.35330353702934114) > weight (0.25) 2021-02-20T10:12:00.526 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T10:12:33.990 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:12:34.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:12:34.022 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:12:34.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:12:34.017+0000 7f1de253f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42777) UID: 1000 2021-02-20T10:12:34.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:12:42.023 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:12:42.024 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:12:42.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:12:42.025 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:12:42.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:12:42.025 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:12:42.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:12:42.028 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T10:13:19.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33163275021600735) > weight (0.25) 2021-02-20T10:13:19.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4227155137755736) > weight (0.25) 2021-02-20T10:13:19.538 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:13:19.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:13:19.539 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:13:19.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:13:19.536+0000 7fa59bca7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 42108) UID: 1000 2021-02-20T10:13:19.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:13:21.540 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:13:21.544 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:13:21.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:13:21.547 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:13:21.547 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:13:21.549 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:13:21.562 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:13:21.563 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T10:14:04.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.37804109226312343) > weight (0.25) 2021-02-20T10:14:04.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9478468789886402) > weight (0.25) 2021-02-20T10:14:04.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7668533459787693) > weight (0.25) 2021-02-20T10:14:04.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8501025704870091) > weight (0.25) 2021-02-20T10:14:04.039 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T10:15:16.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6985961244528162) > weight (0.25) 2021-02-20T10:15:16.599 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5655101104680531) > weight (0.25) 2021-02-20T10:15:16.599 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:15:16.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:15:16.601 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:15:16.601 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:15:16.601 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:15:16.602 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:15:16.597+0000 7ff4ec758700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43067) UID: 1000 2021-02-20T10:15:17.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:15:22.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:15:22.703 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:15:22.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:15:22.704 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:15:22.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:15:22.707 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T10:15:23.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7509046015210104) > weight (0.25) 2021-02-20T10:15:23.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3717521139387495) > weight (0.25) 2021-02-20T10:15:23.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4968675363391114) > weight (0.25) 2021-02-20T10:15:23.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8588525790049312) > weight (0.25) 2021-02-20T10:15:23.042 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T10:16:37.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4736878107444594) > weight (0.25) 2021-02-20T10:16:37.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9524413891478705) > weight (0.25) 2021-02-20T10:16:37.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.48440163405652437) > weight (0.25) 2021-02-20T10:16:37.709 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:16:37.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:16:37.711 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:16:37.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T10:16:37.708+0000 7fea15685700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39005) UID: 1000 2021-02-20T10:16:38.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:16:39.711 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:16:39.712 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:16:39.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:16:39.713 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:16:39.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:16:39.713 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:16:39.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:16:39.717 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T10:16:52.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3480842552991201) > weight (0.25) 2021-02-20T10:16:52.050 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25265321488394665) > weight (0.25) 2021-02-20T10:16:52.050 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6053097565176264) > weight (0.25) 2021-02-20T10:16:52.050 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T10:16:52.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T10:16:52.052 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:16:52.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T10:16:52.051+0000 7f10510c1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41717) UID: 1000 2021-02-20T10:16:52.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:16:53.053 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T10:16:53.053 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:16:53.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:16:53.054 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T10:16:53.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T10:16:53.055 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T10:16:53.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T10:16:53.058 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:18:09.687 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:18:09.680+0000 7f508e15e700 -1 reset not still connected to 0x55ac862ca700 2021-02-20T10:18:09.687 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:18:09.680+0000 7f508e15e700 -1 reset not still connected to 0x55ac862caa40 2021-02-20T10:18:09.688 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:18:09.680+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f9190 2021-02-20T10:18:09.688 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:18:09.680+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133aa90 2021-02-20T10:18:21.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7177781219559741) > weight (0.25) 2021-02-20T10:18:21.729 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:18:21.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:18:21.731 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:18:21.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:18:21.733+0000 7f0562ae0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 42684) UID: 1000 2021-02-20T10:18:22.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:18:24.731 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:18:24.741 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:18:24.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:18:24.742 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:18:24.743 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:18:24.743 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:18:24.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:18:24.747 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T10:18:44.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7324382548093344) > weight (0.25) 2021-02-20T10:18:44.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6767302610871891) > weight (0.25) 2021-02-20T10:18:44.068 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:18:44.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:18:44.069 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:18:44.069 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:18:44.069 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:18:44.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T10:18:44.069+0000 7fc670c61700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 42008) UID: 1000 2021-02-20T10:18:44.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:18:45.538 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:18:45.533+0000 7f803c4bb700 -1 reset not still connected to 0x55acba9a2ff0 2021-02-20T10:18:45.539 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:18:45.533+0000 7f803c4bb700 -1 reset not still connected to 0x55acea49e8f0 2021-02-20T10:18:50.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:18:50.172 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:18:50.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:18:50.172 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:18:50.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:18:50.175 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T10:19:33.747 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:19:33.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:19:33.749 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:19:33.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:19:33.744+0000 7f8de9b1c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42876) UID: 1000 2021-02-20T10:19:34.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:19:41.751 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:19:41.751 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:19:41.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:19:41.752 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:19:41.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:19:41.753 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:19:41.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:19:41.756 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T10:20:06.316 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:20:06.313+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e23dc0 2021-02-20T10:20:17.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5393089276190134) > weight (0.25) 2021-02-20T10:20:17.186 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3567649850261) > weight (0.25) 2021-02-20T10:20:17.187 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48559995528765354) > weight (0.25) 2021-02-20T10:20:17.187 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4932029125132331) > weight (0.25) 2021-02-20T10:20:17.187 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T10:21:14.757 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:21:14.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:21:14.765 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:21:14.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:21:14.762+0000 7f6e3b479700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43639) UID: 1000 2021-02-20T10:21:14.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:21:15.766 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:21:15.766 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:21:15.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:21:15.768 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:21:15.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:21:15.769 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:21:15.784 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:21:15.784 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T10:22:11.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:22:11.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:22:11.197 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:22:11.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:22:11.192+0000 7f54c3020700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42968) UID: 1000 2021-02-20T10:22:11.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:22:14.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:22:14.198 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:22:14.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:22:14.199 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:22:14.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:22:14.199 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:22:14.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:22:14.203 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:22:59.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8417988904475546) > weight (0.25) 2021-02-20T10:22:59.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.71482321990064) > weight (0.25) 2021-02-20T10:22:59.824 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:22:59.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:22:59.825 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:22:59.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:22:59.819+0000 7f16a8328700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43163) UID: 1000 2021-02-20T10:23:00.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:23:05.826 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:23:05.828 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:23:05.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:23:05.849 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:23:05.856 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:23:05.869 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:23:05.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:23:05.924 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T10:23:55.006 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:23:55.000+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b991e0 2021-02-20T10:24:10.204 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:24:10.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:24:10.213 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:24:10.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:24:10.210+0000 7fa39a0ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 43825) UID: 1000 2021-02-20T10:24:10.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:24:14.215 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:24:14.221 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:24:14.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:24:14.223 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:24:14.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:24:14.223 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:24:14.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:24:14.229 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T10:24:21.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34901017055729056) > weight (0.25) 2021-02-20T10:24:21.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29236217550870547) > weight (0.25) 2021-02-20T10:24:21.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4762085480143663) > weight (0.25) 2021-02-20T10:24:21.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4445299241318301) > weight (0.25) 2021-02-20T10:24:21.935 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T10:25:25.006 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:25:25.002+0000 7f508e15e700 -1 reset not still connected to 0x55ac846e6c30 2021-02-20T10:25:25.006 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:25:25.002+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b991e0 2021-02-20T10:25:30.936 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:25:30.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:25:30.945 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:25:30.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:25:30.944+0000 7f7ce66f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43734) UID: 1000 2021-02-20T10:25:31.347 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:25:38.946 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:25:38.953 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:25:38.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:25:38.954 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:25:38.954 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:25:38.955 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:25:38.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:25:38.958 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T10:25:54.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6383539520364163) > weight (0.25) 2021-02-20T10:25:54.249 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:25:54.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:25:54.251 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:25:54.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:25:54.248+0000 7fa5399c8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42585) UID: 1000 2021-02-20T10:25:54.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:25:57.252 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:25:57.261 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:25:57.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:25:57.262 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:25:57.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:25:57.262 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:25:57.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:25:57.272 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T10:26:38.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3535483430048638) > weight (0.25) 2021-02-20T10:26:38.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6937844008583967) > weight (0.25) 2021-02-20T10:26:38.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9510935910875319) > weight (0.25) 2021-02-20T10:26:38.971 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:26:38.971 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:26:38.972 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:26:38.973 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T10:26:38.971+0000 7fcd076e1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 43255) UID: 1000 2021-02-20T10:26:39.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:26:41.973 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:26:41.980 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:26:41.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:26:41.981 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:26:41.981 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:26:41.982 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:26:41.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:26:41.985 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T10:27:52.274 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9179228474267822) > weight (0.25) 2021-02-20T10:27:53.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3621286851419956) > weight (0.25) 2021-02-20T10:27:53.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6639983502310961) > weight (0.25) 2021-02-20T10:27:53.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.43325381173477073) > weight (0.25) 2021-02-20T10:27:53.209 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T10:28:07.986 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:28:07.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:28:07.994 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:28:07.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:28:07.989+0000 7fecf3766700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44119) UID: 1000 2021-02-20T10:28:08.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:28:12.995 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:28:12.995 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:28:12.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:28:12.996 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:28:12.996 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:28:12.996 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:28:13.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:28:13.001 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T10:29:11.211 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:29:11.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:29:11.219 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:29:11.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:29:11.215+0000 7f4323323700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44015) UID: 1000 2021-02-20T10:29:11.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:29:13.220 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:29:13.228 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:29:13.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:29:13.229 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:29:13.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:29:13.230 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:29:13.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:29:13.233 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T10:30:09.002 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:30:09.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:30:09.998 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:30:09.999 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:30:09.994+0000 7f2e96c95700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44401) UID: 1000 2021-02-20T10:30:10.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:30:14.998 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:30:15.006 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:30:15.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:30:15.007 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:30:15.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:30:15.008 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:30:15.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:30:15.012 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T10:30:45.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8246737294194157) > weight (0.25) 2021-02-20T10:30:45.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8821406603196424) > weight (0.25) 2021-02-20T10:30:45.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7629395779326205) > weight (0.25) 2021-02-20T10:30:45.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8063791764365142) > weight (0.25) 2021-02-20T10:30:45.243 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T10:31:32.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8816982623162989) > weight (0.25) 2021-02-20T10:31:32.020 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:31:32.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:31:32.021 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:31:32.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:31:32.016+0000 7f52a93a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 43448) UID: 1000 2021-02-20T10:31:32.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:31:39.022 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:31:39.030 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:31:39.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:31:39.031 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:31:39.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:31:39.031 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:31:39.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:31:39.035 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T10:32:05.441 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:32:05.432+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3a750 2021-02-20T10:32:42.244 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:32:42.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:32:42.275 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:32:42.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:32:42.275+0000 7f2383719700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44493) UID: 1000 2021-02-20T10:32:42.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:32:50.276 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:32:50.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:32:50.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:32:50.287 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:32:50.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:32:50.288 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:32:50.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:32:50.293 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T10:33:25.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4929384704145636) > weight (0.25) 2021-02-20T10:33:25.045 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:33:25.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:33:25.046 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:33:25.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:33:25.042+0000 7fd4d0d4d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 44687) UID: 1000 2021-02-20T10:33:25.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:33:28.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:33:28.057 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:33:28.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:33:28.058 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:33:28.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:33:28.059 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:33:28.061 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:33:28.062 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T10:34:20.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7483517339883071) > weight (0.25) 2021-02-20T10:34:20.295 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8296496949828112) > weight (0.25) 2021-02-20T10:34:20.296 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:34:20.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:34:20.297 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:34:20.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T10:34:20.293+0000 7fdb4e1e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 43540) UID: 1000 2021-02-20T10:34:20.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:34:27.298 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:34:27.299 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:34:27.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:34:27.300 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:34:27.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:34:27.300 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:34:27.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:34:27.304 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T10:34:53.195 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:34:53.188+0000 7f803c4bb700 -1 reset not still connected to 0x55acea49e8f0 2021-02-20T10:35:13.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6125799311756887) > weight (0.25) 2021-02-20T10:35:13.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5020042493264157) > weight (0.25) 2021-02-20T10:35:13.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33685810330024046) > weight (0.25) 2021-02-20T10:35:13.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6639434996884677) > weight (0.25) 2021-02-20T10:35:13.072 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:35:31.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6495084367258882) > weight (0.25) 2021-02-20T10:35:31.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43093322879445983) > weight (0.25) 2021-02-20T10:35:31.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6880331678869533) > weight (0.25) 2021-02-20T10:35:31.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4627449209763761) > weight (0.25) 2021-02-20T10:35:31.307 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T10:36:54.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47611474868052495) > weight (0.25) 2021-02-20T10:36:54.321 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:36:54.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:36:54.323 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:36:54.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:36:54.317+0000 7fb282248700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44210) UID: 1000 2021-02-20T10:36:54.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:37:00.324 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:37:00.330 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:37:00.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:37:00.331 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:37:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:37:00.332 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:37:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:37:00.335 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T10:37:04.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.44118364842596736) > weight (0.25) 2021-02-20T10:37:04.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9167600395388442) > weight (0.25) 2021-02-20T10:37:04.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26300864855249206) > weight (0.25) 2021-02-20T10:37:04.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5432639846431538) > weight (0.25) 2021-02-20T10:37:04.078 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T10:37:22.291 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:37:22.285+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3ac30 2021-02-20T10:37:22.291 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:37:22.285+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3d810 2021-02-20T10:38:01.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4919571152553567) > weight (0.25) 2021-02-20T10:38:01.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8623934620660574) > weight (0.25) 2021-02-20T10:38:01.338 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:38:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:38:01.339 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:38:01.340 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:38:01.340 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:38:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T10:38:01.336+0000 7f4e531de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 44969) UID: 1000 2021-02-20T10:38:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:38:07.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:38:07.461 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:38:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:38:07.462 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:38:07.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:38:07.465 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T10:38:31.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8660344882155405) > weight (0.25) 2021-02-20T10:38:31.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5588019064010288) > weight (0.25) 2021-02-20T10:38:31.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9127800651074254) > weight (0.25) 2021-02-20T10:38:31.087 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:38:31.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:38:31.088 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:38:31.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T10:38:31.083+0000 7ff183030700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 44310) UID: 1000 2021-02-20T10:38:31.490 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:38:35.089 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:38:35.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:38:35.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:38:35.090 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:38:35.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:38:35.091 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:38:35.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:38:35.094 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T10:39:44.466 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:39:44.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:39:44.468 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T10:39:44.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:39:44.466+0000 7f6613381700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44779) UID: 1000 2021-02-20T10:39:44.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:39:51.468 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:39:51.469 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:39:51.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:39:51.470 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:39:51.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:39:51.471 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:39:51.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:39:51.474 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T10:40:00.095 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:40:00.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:40:00.096 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:40:00.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:40:00.093+0000 7ff031a4e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44594) UID: 1000 2021-02-20T10:40:00.498 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:40:08.097 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:40:08.097 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:40:08.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:40:08.098 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:40:08.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:40:08.099 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:40:08.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:40:08.102 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T10:41:08.103 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:41:08.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:41:08.111 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T10:41:08.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:41:08.108+0000 7fbe36e7e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45465) UID: 1000 2021-02-20T10:41:08.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:41:09.114 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:41:09.114 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:41:09.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:41:09.116 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:41:09.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:41:09.116 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:41:09.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:41:09.124 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T10:41:48.475 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3692512065449478) > weight (0.25) 2021-02-20T10:41:48.483 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9220298044257067) > weight (0.25) 2021-02-20T10:41:48.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5921411658954626) > weight (0.25) 2021-02-20T10:41:48.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5051265466304994) > weight (0.25) 2021-02-20T10:41:48.484 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T10:42:43.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6806740236856492) > weight (0.25) 2021-02-20T10:42:43.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.952201544114778) > weight (0.25) 2021-02-20T10:42:43.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5012126715981033) > weight (0.25) 2021-02-20T10:42:43.137 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T10:42:43.138 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T10:42:43.139 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:42:43.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T10:42:43.134+0000 7f0412ee2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 45273) UID: 1000 2021-02-20T10:42:43.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:42:49.140 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T10:42:49.148 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:42:49.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:42:49.149 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T10:42:49.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T10:42:49.150 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T10:42:49.154 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T10:42:49.155 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T10:43:07.485 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:43:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:43:07.490 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:43:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:43:07.489+0000 7fb261991700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45365) UID: 1000 2021-02-20T10:43:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:43:11.492 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:43:11.492 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:43:11.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:43:11.493 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:43:11.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:43:11.494 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:43:11.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:43:11.496 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T10:43:38.697 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:43:38.688+0000 7f508e15e700 -1 reset not still connected to 0x55ac6ab78080 2021-02-20T10:43:38.698 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:43:38.688+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c96c0 2021-02-20T10:43:38.698 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:43:38.688+0000 7f508e15e700 -1 reset not still connected to 0x55ac87972a90 2021-02-20T10:43:38.699 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:43:38.688+0000 7f508e15e700 -1 reset not still connected to 0x55ac879736c0 2021-02-20T10:44:06.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7004567441771379) > weight (0.25) 2021-02-20T10:44:06.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.42248605583752885) > weight (0.25) 2021-02-20T10:44:06.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8658131366718831) > weight (0.25) 2021-02-20T10:44:06.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9169598260798132) > weight (0.25) 2021-02-20T10:44:06.166 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T10:44:57.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9929359419020682) > weight (0.25) 2021-02-20T10:44:57.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5709678438476284) > weight (0.25) 2021-02-20T10:44:57.503 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T10:44:57.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T10:44:57.504 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:44:57.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T10:44:57.499+0000 7efffb700700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 45174) UID: 1000 2021-02-20T10:44:57.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:45:00.505 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T10:45:00.511 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:45:00.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:45:00.512 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T10:45:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T10:45:00.513 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T10:45:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T10:45:00.516 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T10:45:46.167 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:45:46.178 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:45:46.178 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:45:46.179 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:45:46.174+0000 7fac4994c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45557) UID: 1000 2021-02-20T10:45:46.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:45:51.179 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:45:51.185 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:45:51.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:45:51.186 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:45:51.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:45:51.187 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:45:51.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:45:51.190 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:46:23.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5377729348614325) > weight (0.25) 2021-02-20T10:46:23.519 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:46:23.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:46:23.520 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:46:23.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:46:23.517+0000 7f453d70f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45075) UID: 1000 2021-02-20T10:46:23.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:46:28.521 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:46:28.522 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:46:28.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:46:28.523 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:46:28.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:46:28.523 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:46:28.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:46:28.526 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T10:46:58.191 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:46:58.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:46:58.192 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:46:58.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:46:58.188+0000 7fe311982700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46009) UID: 1000 2021-02-20T10:46:58.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:47:04.193 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:47:04.201 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:47:04.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:47:04.202 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:47:04.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:47:04.203 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:47:04.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:47:04.208 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T10:47:38.528 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:47:38.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:47:38.530 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:47:38.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:47:38.528+0000 7fa3c5439700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45741) UID: 1000 2021-02-20T10:47:38.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:47:41.531 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:47:41.532 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:47:41.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:47:41.533 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:47:41.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:47:41.534 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:47:41.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:47:41.537 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T10:48:55.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4264013261050158) > weight (0.25) 2021-02-20T10:48:55.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8883811780091674) > weight (0.25) 2021-02-20T10:48:55.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3734544590171167) > weight (0.25) 2021-02-20T10:48:55.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.41517329886617294) > weight (0.25) 2021-02-20T10:48:55.220 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T10:49:14.538 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:49:14.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:49:14.539 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T10:49:14.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:49:14.537+0000 7f4946fcc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46293) UID: 1000 2021-02-20T10:49:14.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:49:22.541 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:49:23.018 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:49:23.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:49:23.021 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:49:23.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:49:23.022 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:49:23.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:49:23.036 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T10:49:58.326 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:49:58.321+0000 7f803c4bb700 -1 reset not still connected to 0x55acb88ed110 2021-02-20T10:49:58.327 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:49:58.321+0000 7f803c4bb700 -1 reset not still connected to 0x55acba9a3260 2021-02-20T10:49:58.327 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:49:58.321+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b29c0 2021-02-20T10:50:13.221 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5846346115224085) > weight (0.25) 2021-02-20T10:50:13.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36719807927100356) > weight (0.25) 2021-02-20T10:50:13.222 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T10:50:13.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T10:50:13.223 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T10:50:13.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:50:13.220+0000 7f5fa8d7e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 43924) UID: 1000 2021-02-20T10:50:13.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:50:19.225 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T10:50:19.237 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:50:19.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:50:19.239 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T10:50:19.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T10:50:19.240 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T10:50:19.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T10:50:19.245 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T10:51:09.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8441044327045089) > weight (0.25) 2021-02-20T10:51:09.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8141470403234727) > weight (0.25) 2021-02-20T10:51:09.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7537375906956325) > weight (0.25) 2021-02-20T10:51:09.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8572394062307069) > weight (0.25) 2021-02-20T10:51:09.039 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T10:52:05.246 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:52:05.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:52:05.259 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:52:05.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:52:05.254+0000 7fb832160700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46200) UID: 1000 2021-02-20T10:52:05.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:52:07.260 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:52:07.261 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:52:07.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:52:07.262 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:52:07.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:52:07.262 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:52:07.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:52:07.265 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T10:52:33.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44588534131397417) > weight (0.25) 2021-02-20T10:52:33.040 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:52:33.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:52:33.041 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T10:52:33.042 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:52:33.042 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:52:33.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:52:33.037+0000 7fd95c9f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46100) UID: 1000 2021-02-20T10:52:33.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:52:39.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:52:39.144 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:52:39.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:52:39.144 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:52:39.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:52:39.147 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T10:53:35.271 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:53:35.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:53:35.274 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T10:53:35.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:53:35.268+0000 7f01894f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46585) UID: 1000 2021-02-20T10:53:35.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:53:40.274 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:53:40.275 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:53:40.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:53:40.276 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:53:40.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:53:40.276 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:53:40.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:53:40.280 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T10:54:21.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7389009730541599) > weight (0.25) 2021-02-20T10:54:21.150 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:54:21.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:54:21.152 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:54:21.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:54:21.147+0000 7f1ea990d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46677) UID: 1000 2021-02-20T10:54:21.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:54:30.153 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:54:30.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:54:30.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:54:30.154 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:54:30.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:54:30.155 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:54:30.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:54:30.158 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T10:55:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8799620363298917) > weight (0.25) 2021-02-20T10:55:18.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5655778658954744) > weight (0.25) 2021-02-20T10:55:18.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9155190614013183) > weight (0.25) 2021-02-20T10:55:18.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6191065033781796) > weight (0.25) 2021-02-20T10:55:18.283 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T10:55:53.451 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:55:53.445+0000 7f803c4bb700 -1 reset not still connected to 0x55aca6c58410 2021-02-20T10:55:53.452 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:55:53.445+0000 7f803c4bb700 -1 reset not still connected to 0x55acb4af85b0 2021-02-20T10:55:53.452 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:55:53.445+0000 7f803c4bb700 -1 reset not still connected to 0x55acbadbda00 2021-02-20T10:55:53.452 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:55:53.445+0000 7f803c4bb700 -1 reset not still connected to 0x55acd59084e0 2021-02-20T10:55:54.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8989449686952512) > weight (0.25) 2021-02-20T10:55:54.160 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:55:54.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:55:54.161 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:55:54.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:55:54.157+0000 7f74e77f9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46870) UID: 1000 2021-02-20T10:55:54.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:55:58.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:55:58.162 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:55:58.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:55:58.163 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:55:58.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:55:58.164 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:55:58.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:55:58.167 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T10:56:18.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5196682080028522) > weight (0.25) 2021-02-20T10:56:18.284 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T10:56:18.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T10:56:18.284 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T10:56:18.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T10:56:18.280+0000 7f8069a00700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 44878) UID: 1000 2021-02-20T10:56:18.686 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:56:19.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.512+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4ec80 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T10:56:19.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.512+0000 7faacfb07700 -1 rbd::mirror::ImageReplayer: 0x564854e4ef00 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T10:56:19.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4ec80 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T10:56:19.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4ef00 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T10:56:19.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4f680 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T10:56:19.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4fe00 [3/4c4f4929-06c7-4203-81c6-a0c9560ab291] start: already running 2021-02-20T10:56:19.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e50080 [3/4d947326-1304-4f75-9cb5-1cc9895ee2c8] start: already running 2021-02-20T10:56:19.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4f180 [3/674f23ca-7630-45f2-9a82-095e98b79c76] start: already running 2021-02-20T10:56:19.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e50300 [3/9a469893-68d6-4d91-a371-3899fcc47454] start: already running 2021-02-20T10:56:19.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e50580 [3/b5d77140-8b49-4567-b4bc-f2d77cea0806] start: already running 2021-02-20T10:56:19.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T10:56:19.520+0000 7faad0308700 -1 rbd::mirror::ImageReplayer: 0x564854e4f400 [3/d83b5ea1-0f06-4764-9c22-8791c64dfe81] start: already running 2021-02-20T10:56:24.339 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:56:24.334+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3d810 2021-02-20T10:56:24.339 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:56:24.334+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b21670 2021-02-20T10:56:24.342 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:56:24.338+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e22970 2021-02-20T10:56:24.342 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T10:56:24.338+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e22ff0 2021-02-20T10:56:27.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T10:56:27.286 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:56:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:56:27.287 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T10:56:27.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T10:56:27.287 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T10:56:27.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T10:56:27.291 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T10:57:32.292 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:57:32.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:57:32.294 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T10:57:32.295 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:57:32.291+0000 7f6cde282700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 46779) UID: 1000 2021-02-20T10:57:32.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:57:34.295 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:57:34.296 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:57:34.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:57:34.297 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:57:34.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:57:34.298 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:57:34.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:57:34.301 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T10:57:53.168 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T10:57:53.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T10:57:53.178 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:57:53.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T10:57:53.174+0000 7fa118704700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46394) UID: 1000 2021-02-20T10:57:53.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:57:56.179 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T10:57:56.187 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:57:56.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:57:56.188 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T10:57:56.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T10:57:56.189 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T10:57:56.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T10:57:56.194 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T10:58:41.315 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T10:58:41.309+0000 7ff4f84bf700 -1 reset not still connected to 0x55db2ca39cf0 2021-02-20T10:58:41.316 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T10:58:41.309+0000 7ff4f84bf700 -1 reset not still connected to 0x55db320c8820 2021-02-20T10:58:41.316 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T10:58:41.309+0000 7ff4f84bf700 -1 reset not still connected to 0x55db8f2fe8f0 2021-02-20T10:58:53.303 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T10:58:53.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T10:58:53.316 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T10:58:53.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T10:58:53.309+0000 7f5e77cb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47163) UID: 1000 2021-02-20T10:58:53.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:58:57.317 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T10:58:57.333 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:58:57.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:58:57.334 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T10:58:57.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T10:58:57.335 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T10:58:57.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T10:58:57.338 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T10:59:22.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6176534375505) > weight (0.25) 2021-02-20T10:59:22.196 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T10:59:22.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T10:59:22.197 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T10:59:22.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T10:59:22.192+0000 7f8bb833b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46972) UID: 1000 2021-02-20T10:59:22.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T10:59:25.198 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T10:59:25.199 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T10:59:25.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T10:59:25.200 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T10:59:25.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T10:59:25.200 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T10:59:25.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T10:59:25.203 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T10:59:27.377 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:59:27.368+0000 7f803c4bb700 -1 reset not still connected to 0x55accab03810 2021-02-20T10:59:27.377 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T10:59:27.368+0000 7f803c4bb700 -1 reset not still connected to 0x55acea49e8f0 2021-02-20T11:00:24.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.642889346105057) > weight (0.25) 2021-02-20T11:00:24.386 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:00:24.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:00:24.388 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:00:24.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:00:24.383+0000 7f4b0c192700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 47071) UID: 1000 2021-02-20T11:00:24.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:00:28.389 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:00:28.390 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:00:28.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:00:28.390 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:00:28.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:00:28.391 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:00:28.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:00:28.394 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T11:00:32.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7668753970825712) > weight (0.25) 2021-02-20T11:00:32.205 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:00:32.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:00:32.206 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:00:32.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:00:32.199+0000 7f11221e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47446) UID: 1000 2021-02-20T11:00:32.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:00:41.207 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:00:41.208 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:00:41.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:00:41.209 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:00:41.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:00:41.210 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:00:41.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:00:41.213 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T11:01:50.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7819766141354719) > weight (0.25) 2021-02-20T11:01:50.404 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:01:50.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:01:50.406 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:01:50.407 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:01:50.401+0000 7f39e8a9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 47546) UID: 1000 2021-02-20T11:01:51.407 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:01:51.408 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:01:51.809 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:01:57.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:01:57.508 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:01:57.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:01:57.509 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:01:57.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:01:57.513 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T11:02:27.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4069289460665938) > weight (0.25) 2021-02-20T11:02:27.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6932067102601117) > weight (0.25) 2021-02-20T11:02:27.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9543965858027459) > weight (0.25) 2021-02-20T11:02:27.216 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9570654098571904) > weight (0.25) 2021-02-20T11:02:27.216 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T11:03:31.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5701136037577986) > weight (0.25) 2021-02-20T11:03:31.218 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6721796821558715) > weight (0.25) 2021-02-20T11:03:31.218 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:03:31.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:03:31.219 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:03:31.220 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:03:31.220 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:03:31.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:03:31.215+0000 7fe71283d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 45907) UID: 1000 2021-02-20T11:03:31.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:03:37.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:03:37.570 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:03:37.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:03:37.571 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:03:37.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:03:37.574 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T11:03:52.513 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3596672955337161) > weight (0.25) 2021-02-20T11:03:52.514 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9319398723662701) > weight (0.25) 2021-02-20T11:03:52.514 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.954982390953585) > weight (0.25) 2021-02-20T11:03:52.514 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38377684529863176) > weight (0.25) 2021-02-20T11:03:52.514 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T11:04:10.415 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:04:10.407+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3d810 2021-02-20T11:05:10.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2534913104021831) > weight (0.25) 2021-02-20T11:05:10.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4403152208439308) > weight (0.25) 2021-02-20T11:05:10.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5266642526996509) > weight (0.25) 2021-02-20T11:05:10.517 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8205292534998603) > weight (0.25) 2021-02-20T11:05:10.517 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T11:05:33.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6390542456058368) > weight (0.25) 2021-02-20T11:05:33.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9353986242860767) > weight (0.25) 2021-02-20T11:05:33.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5112214305989076) > weight (0.25) 2021-02-20T11:05:33.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5384908220901317) > weight (0.25) 2021-02-20T11:05:33.990 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T11:07:02.518 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5489418333819911) > weight (0.25) 2021-02-20T11:07:02.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6587489327984317) > weight (0.25) 2021-02-20T11:07:02.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2505525427800518) > weight (0.25) 2021-02-20T11:07:02.528 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:07:02.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:07:02.530 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:07:02.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:07:02.526+0000 7fd4b1975700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 45650) UID: 1000 2021-02-20T11:07:02.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:07:08.530 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:07:08.531 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:07:08.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:07:08.532 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:07:08.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:07:08.532 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:07:08.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:07:08.535 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T11:07:11.991 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4122507729769852) > weight (0.25) 2021-02-20T11:07:11.991 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:07:11.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:07:11.992 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:07:11.992 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:07:11.992 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:07:11.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:07:11.986+0000 7f2aaf570700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47637) UID: 1000 2021-02-20T11:07:12.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:07:18.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:07:18.095 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:07:18.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:07:18.095 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:07:18.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:07:18.098 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T11:07:57.769 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:07:57.761+0000 7f508e15e700 -1 reset not still connected to 0x55ac846e9dc0 2021-02-20T11:07:57.770 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:07:57.761+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b21e90 2021-02-20T11:08:18.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5243804303070522) > weight (0.25) 2021-02-20T11:08:18.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5926000685232177) > weight (0.25) 2021-02-20T11:08:18.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2706724897100573) > weight (0.25) 2021-02-20T11:08:18.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.27183755287998457) > weight (0.25) 2021-02-20T11:08:18.101 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T11:08:37.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46270380738279027) > weight (0.25) 2021-02-20T11:08:37.537 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:08:37.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:08:37.539 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:08:37.539 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:08:37.540 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:08:37.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:08:37.532+0000 7fc39dc4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 47736) UID: 1000 2021-02-20T11:08:37.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:08:38.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.800+0000 7f8fbe6f9700 -1 rbd::mirror::ImageReplayer: 0x55656f6c6c80 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T11:08:38.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.800+0000 7f8fbe6f9700 -1 rbd::mirror::ImageReplayer: 0x55656f6c7180 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T11:08:38.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbe6f9700 -1 rbd::mirror::ImageReplayer: 0x55656f6c7400 [3/3ca6eed3-b4cc-42a3-8669-fd637c7a8d69] start: already running 2021-02-20T11:08:38.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbeefa700 -1 rbd::mirror::ImageReplayer: 0x55656f6c7e00 [3/75792560-55ec-454a-bb3c-f7a23e6221d8] start: already running 2021-02-20T11:08:38.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbeefa700 -1 rbd::mirror::ImageReplayer: 0x55656f6c7b80 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T11:08:38.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbeefa700 -1 rbd::mirror::ImageReplayer: 0x55656f6c8080 [3/7cc83948-bf26-4ed7-a142-2db0f2786e6e] start: already running 2021-02-20T11:08:38.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbeefa700 -1 rbd::mirror::ImageReplayer: 0x55656f6c7900 [3/828e1151-fa83-405f-a6cf-ec7c06ae0dee] start: already running 2021-02-20T11:08:38.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:08:38.804+0000 7f8fbeefa700 -1 rbd::mirror::ImageReplayer: 0x55656f6c8580 [3/c5428c9a-3b56-4be4-96e7-515eb901b226] start: already running 2021-02-20T11:08:43.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:08:43.642 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:08:43.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:08:43.643 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:08:43.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:08:43.646 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T11:09:54.647 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:09:54.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:09:54.648 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:09:54.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:09:54.643+0000 7fc8568f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47355) UID: 1000 2021-02-20T11:09:55.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:09:57.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:09:57.650 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:09:57.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:09:57.651 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:09:57.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:09:57.652 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:09:57.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:09:57.659 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:10:08.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7744972535764417) > weight (0.25) 2021-02-20T11:10:08.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8250673542037572) > weight (0.25) 2021-02-20T11:10:08.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6734730968996069) > weight (0.25) 2021-02-20T11:10:08.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39145770056392837) > weight (0.25) 2021-02-20T11:10:08.103 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T11:11:37.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25744070993272206) > weight (0.25) 2021-02-20T11:11:37.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5288915274042547) > weight (0.25) 2021-02-20T11:11:37.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8561654623945504) > weight (0.25) 2021-02-20T11:11:37.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5658398217094812) > weight (0.25) 2021-02-20T11:11:37.662 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T11:12:07.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4537021843226141) > weight (0.25) 2021-02-20T11:12:07.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36144010635997426) > weight (0.25) 2021-02-20T11:12:07.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8459649761671054) > weight (0.25) 2021-02-20T11:12:07.105 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:12:07.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:12:07.106 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:12:07.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T11:12:07.100+0000 7f04c3c0c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 43346) UID: 1000 2021-02-20T11:12:07.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:12:15.108 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:12:15.108 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:12:15.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:12:15.110 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:12:15.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:12:15.110 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:12:15.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:12:15.114 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T11:13:19.663 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:13:19.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:13:19.666 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:13:19.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:13:19.662+0000 7fdf0d348700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48219) UID: 1000 2021-02-20T11:13:21.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:13:22.116 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:13:22.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:13:22.117 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:13:22.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:13:22.110+0000 7f46a6d04700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47254) UID: 1000 2021-02-20T11:13:22.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:13:22.667 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:13:22.667 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:13:22.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:13:22.668 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:13:22.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:13:22.669 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:13:22.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:13:22.672 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T11:13:26.118 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:13:26.119 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:13:26.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:13:26.119 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:13:26.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:13:26.120 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:13:26.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:13:26.123 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T11:13:28.364 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:13:28.358+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f0f22f0 2021-02-20T11:13:28.364 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:13:28.358+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c8820 2021-02-20T11:13:28.365 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:13:28.358+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f7ba0 2021-02-20T11:15:03.673 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:15:03.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:15:03.674 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:15:03.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:15:03.668+0000 7f98fcf9e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48412) UID: 1000 2021-02-20T11:15:04.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:15:05.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:15:05.676 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:15:05.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:15:05.677 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:15:05.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:15:05.678 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:15:05.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:15:05.692 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T11:15:23.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7460710979962194) > weight (0.25) 2021-02-20T11:15:23.124 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:15:23.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:15:23.125 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:15:23.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:15:23.120+0000 7f57153ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48024) UID: 1000 2021-02-20T11:15:23.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:15:26.126 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:15:26.126 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:15:26.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:15:26.127 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:15:26.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:15:26.128 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:15:26.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:15:26.131 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:15:54.690 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f0f22f0 2021-02-20T11:15:54.691 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac81677cf0 2021-02-20T11:15:54.691 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c8820 2021-02-20T11:15:54.692 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac872728f0 2021-02-20T11:15:54.692 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac87274e50 2021-02-20T11:15:54.692 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac879729c0 2021-02-20T11:15:54.692 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f7520 2021-02-20T11:15:54.692 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f7ba0 2021-02-20T11:15:54.693 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:15:54.682+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e20270 2021-02-20T11:16:47.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5623587810353062) > weight (0.25) 2021-02-20T11:16:47.694 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:16:47.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:16:47.695 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:16:47.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:16:47.690+0000 7fd1a0d15700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48124) UID: 1000 2021-02-20T11:16:48.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:16:52.696 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:16:52.697 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:16:52.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:16:52.698 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:16:52.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:16:52.699 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:16:52.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:16:52.702 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T11:17:06.132 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8479260480410047) > weight (0.25) 2021-02-20T11:17:06.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2868397919212038) > weight (0.25) 2021-02-20T11:17:06.133 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:17:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:17:06.134 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:17:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:17:06.129+0000 7fab5ce66700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47828) UID: 1000 2021-02-20T11:17:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:17:15.135 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:17:15.135 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:17:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:17:15.136 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:17:15.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:17:15.137 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:17:15.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:17:15.140 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T11:18:47.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8293453933811819) > weight (0.25) 2021-02-20T11:18:47.703 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:18:47.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:18:47.704 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:18:47.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:18:47.695+0000 7f7070974700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48798) UID: 1000 2021-02-20T11:18:48.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:18:56.705 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:18:56.705 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:18:56.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:18:56.706 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:18:56.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:18:56.707 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:18:56.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:18:56.710 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T11:19:03.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47076987546285554) > weight (0.25) 2021-02-20T11:19:03.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4574113437540619) > weight (0.25) 2021-02-20T11:19:03.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.34346817097372495) > weight (0.25) 2021-02-20T11:19:03.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9179584672636701) > weight (0.25) 2021-02-20T11:19:03.141 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T11:20:30.711 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:20:30.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:20:30.713 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:20:30.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:20:30.705+0000 7faf26187700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48606) UID: 1000 2021-02-20T11:20:31.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:20:31.142 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:20:31.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:20:31.143 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:20:31.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:20:31.165+0000 7f214623b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48503) UID: 1000 2021-02-20T11:20:31.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:20:32.144 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:20:32.144 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:20:32.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:20:32.145 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:20:32.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:20:32.146 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:20:32.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:20:32.150 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T11:20:36.715 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:20:36.717 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:20:36.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:20:36.719 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:20:36.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:20:36.721 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:20:36.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:20:36.725 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T11:20:37.621 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:20:37.609+0000 7f803c4bb700 -1 reset not still connected to 0x55acb55332b0 2021-02-20T11:20:37.624 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:20:37.617+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9174dd0 2021-02-20T11:20:37.624 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:20:37.617+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9176490 2021-02-20T11:20:37.624 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:20:37.617+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1334b60 2021-02-20T11:20:37.624 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:20:37.617+0000 7f803c4bb700 -1 reset not still connected to 0x55acd590b400 2021-02-20T11:22:14.151 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:22:14.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:22:14.152 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:22:14.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:22:14.147+0000 7fc9f3265700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49087) UID: 1000 2021-02-20T11:22:14.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:22:20.153 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:22:20.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:22:20.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:22:20.154 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:22:20.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:22:20.155 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:22:20.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:22:20.158 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T11:22:32.726 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3854421592172458) > weight (0.25) 2021-02-20T11:22:32.727 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:22:32.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:22:32.727 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:22:32.728 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:22:32.728 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:22:32.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:22:32.722+0000 7fbfc9658700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48994) UID: 1000 2021-02-20T11:22:33.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:22:38.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:22:38.830 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:22:38.830 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:22:38.830 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:22:38.833 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:22:38.833 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T11:22:45.327 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:22:45.322+0000 7f803c4bb700 -1 reset not still connected to 0x55acb91752b0 2021-02-20T11:22:45.329 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:22:45.322+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9176490 2021-02-20T11:22:45.330 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:22:45.322+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1334b60 2021-02-20T11:22:45.330 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:22:45.322+0000 7f803c4bb700 -1 reset not still connected to 0x55acd590b400 2021-02-20T11:23:50.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8255656645989443) > weight (0.25) 2021-02-20T11:23:50.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8521516857493789) > weight (0.25) 2021-02-20T11:23:50.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9205219987927763) > weight (0.25) 2021-02-20T11:23:50.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.38925170318364544) > weight (0.25) 2021-02-20T11:23:50.160 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T11:24:34.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5503363434653857) > weight (0.25) 2021-02-20T11:24:34.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38724875374816925) > weight (0.25) 2021-02-20T11:24:34.835 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:24:34.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:24:34.836 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:24:34.836 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:24:34.837 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:24:34.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T11:24:34.828+0000 7faad0b09700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 46493) UID: 1000 2021-02-20T11:24:35.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:24:40.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:24:40.939 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:24:40.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:24:40.940 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:24:40.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:24:40.944 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T11:24:52.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7657268607280565) > weight (0.25) 2021-02-20T11:24:52.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49439064974581803) > weight (0.25) 2021-02-20T11:24:52.162 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:24:52.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:24:52.163 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:24:52.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:24:52.155+0000 7f9ce8c9e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 48892) UID: 1000 2021-02-20T11:24:52.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:24:56.163 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:24:56.164 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:24:56.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:24:56.164 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:24:56.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:24:56.165 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:24:56.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:24:56.168 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T11:25:41.945 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:25:41.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:25:41.947 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:25:41.947 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:25:41.947 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:25:41.948 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:25:41.942+0000 7fe863490700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49178) UID: 1000 2021-02-20T11:25:42.348 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:25:48.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:25:48.050 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:25:48.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:25:48.051 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:25:48.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:25:48.061 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T11:26:17.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8237261281827568) > weight (0.25) 2021-02-20T11:26:17.170 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:26:17.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:26:17.171 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:26:17.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:26:17.165+0000 7f72fefbd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 48698) UID: 1000 2021-02-20T11:26:17.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:26:21.172 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:26:21.172 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:26:21.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:26:21.173 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:26:21.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:26:21.174 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:26:21.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:26:21.177 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T11:27:14.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8326060136575717) > weight (0.25) 2021-02-20T11:27:14.063 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:27:14.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:27:14.064 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:27:14.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:27:14.056+0000 7fcab7a1c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49377) UID: 1000 2021-02-20T11:27:14.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:27:21.064 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:27:21.065 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:27:21.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:27:21.066 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:27:21.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:27:21.066 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:27:21.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:27:21.069 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T11:27:36.178 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5146906867250186) > weight (0.25) 2021-02-20T11:27:36.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5990830719078303) > weight (0.25) 2021-02-20T11:27:36.190 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:27:36.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:27:36.191 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:27:36.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:27:36.184+0000 7ff59e442700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49560) UID: 1000 2021-02-20T11:27:36.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:27:39.192 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:27:39.254 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:27:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:27:39.255 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:27:39.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:27:39.256 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:27:39.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:27:39.259 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T11:28:57.071 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:28:57.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:28:57.072 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:28:57.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:28:57.066+0000 7f5fd430a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49663) UID: 1000 2021-02-20T11:28:57.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:29:02.073 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:29:02.073 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:29:02.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:29:02.074 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:29:02.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:29:02.075 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:29:02.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:29:02.078 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T11:29:25.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7237559422596926) > weight (0.25) 2021-02-20T11:29:25.261 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:29:25.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:29:25.261 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:29:25.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:29:25.253+0000 7f78ec9ad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 49755) UID: 1000 2021-02-20T11:29:25.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:29:26.263 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:29:26.266 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:29:26.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:29:26.267 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:29:26.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:29:26.267 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:29:26.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:29:26.270 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T11:29:30.847 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:29:30.839+0000 7f508e15e700 -1 reset not still connected to 0x55ac6ab77450 2021-02-20T11:29:30.848 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:29:30.839+0000 7f508e15e700 -1 reset not still connected to 0x55ac87273380 2021-02-20T11:29:30.848 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:29:30.839+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750c630 2021-02-20T11:30:29.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8650480142008438) > weight (0.25) 2021-02-20T11:30:29.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28109495681495544) > weight (0.25) 2021-02-20T11:30:29.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46533564075387723) > weight (0.25) 2021-02-20T11:30:29.080 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5839667838613001) > weight (0.25) 2021-02-20T11:30:29.080 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T11:31:24.272 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9358720458605347) > weight (0.25) 2021-02-20T11:31:24.272 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:31:24.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:31:24.274 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:31:24.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:31:24.267+0000 7f809463b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50135) UID: 1000 2021-02-20T11:31:24.368 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:31:24.360+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750c630 2021-02-20T11:31:24.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:31:29.274 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:31:29.275 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:31:29.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:31:29.276 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:31:29.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:31:29.277 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:31:29.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:31:29.281 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T11:31:35.081 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:31:35.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:31:35.082 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T11:31:35.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:31:35.075+0000 7f2b68dc6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50044) UID: 1000 2021-02-20T11:31:35.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:31:39.083 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:31:39.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:31:39.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:31:39.084 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:31:39.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:31:39.085 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:31:39.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:31:39.088 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T11:32:42.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7223894649633632) > weight (0.25) 2021-02-20T11:32:42.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9704178454048564) > weight (0.25) 2021-02-20T11:32:42.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3833176506674132) > weight (0.25) 2021-02-20T11:32:42.283 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:32:42.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:32:42.284 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:32:42.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T11:32:42.277+0000 7fdae1b9c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 48311) UID: 1000 2021-02-20T11:32:42.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:32:44.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:32:44.285 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:32:44.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:32:44.286 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:32:44.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:32:44.287 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:32:44.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:32:44.289 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T11:33:24.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31602921004108864) > weight (0.25) 2021-02-20T11:33:24.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.512651206157962) > weight (0.25) 2021-02-20T11:33:24.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7595517666861744) > weight (0.25) 2021-02-20T11:33:24.090 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:33:24.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:33:24.091 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:33:24.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:33:24.084+0000 7f8fbf6fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47933) UID: 1000 2021-02-20T11:33:24.315 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T11:33:24.304+0000 7f803c4bb700 -1 reset not still connected to 0x55accf695ad0 2021-02-20T11:33:24.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:33:33.092 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:33:33.092 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:33:33.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:33:33.093 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:33:33.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:33:33.095 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:33:33.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:33:33.098 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T11:34:02.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5205002580510936) > weight (0.25) 2021-02-20T11:34:02.291 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:34:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:34:02.292 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T11:34:02.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:34:02.287+0000 7f670a0ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50238) UID: 1000 2021-02-20T11:34:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:34:09.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:34:09.294 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:34:09.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:34:09.294 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:34:09.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:34:09.295 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:34:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:34:09.303 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T11:34:48.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7118161740930784) > weight (0.25) 2021-02-20T11:34:48.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9416827852546947) > weight (0.25) 2021-02-20T11:34:48.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4948491595082779) > weight (0.25) 2021-02-20T11:34:48.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7494384085451993) > weight (0.25) 2021-02-20T11:34:48.101 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T11:35:39.535 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T11:35:39.525+0000 7fc273bc1700 -1 reset not still connected to 0x56118e31fc70 2021-02-20T11:35:39.536 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T11:35:39.525+0000 7fc273bc1700 -1 reset not still connected to 0x5611a8288700 2021-02-20T11:35:39.536 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T11:35:39.525+0000 7fc273bc1700 -1 reset not still connected to 0x5611af3730c0 2021-02-20T11:35:57.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9109649040216761) > weight (0.25) 2021-02-20T11:35:57.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7749411026311458) > weight (0.25) 2021-02-20T11:35:57.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5050438915380276) > weight (0.25) 2021-02-20T11:35:57.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8462077689004999) > weight (0.25) 2021-02-20T11:35:57.103 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T11:36:03.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4363097191543872) > weight (0.25) 2021-02-20T11:36:03.304 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:36:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:36:03.305 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:36:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:36:03.297+0000 7f24c0976700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50619) UID: 1000 2021-02-20T11:36:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:36:11.306 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:36:11.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:36:11.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:36:11.308 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:36:11.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:36:11.308 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:36:11.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:36:11.312 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T11:37:52.104 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:37:52.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:37:52.106 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:37:52.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:37:52.102+0000 7f3dded88700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50337) UID: 1000 2021-02-20T11:37:52.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:37:58.107 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:37:58.107 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:37:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:37:58.108 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:37:58.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:37:58.109 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:37:58.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:37:58.112 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T11:38:04.313 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:38:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:38:04.314 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:38:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:38:04.306+0000 7fe86c982700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49278) UID: 1000 2021-02-20T11:38:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:38:13.317 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:38:13.317 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:38:13.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:38:13.318 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:38:13.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:38:13.318 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:38:13.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:38:13.322 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T11:38:15.625 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:38:15.619+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b1f930 2021-02-20T11:38:15.626 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:38:15.619+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b208a0 2021-02-20T11:38:15.626 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:38:15.619+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f8630 2021-02-20T11:38:15.628 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:38:15.619+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e20820 2021-02-20T11:39:19.324 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:39:19.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:39:19.330 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:39:19.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:39:19.324+0000 7f2e66d4e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50914) UID: 1000 2021-02-20T11:39:19.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:39:21.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8579254205370166) > weight (0.25) 2021-02-20T11:39:21.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9159017856763972) > weight (0.25) 2021-02-20T11:39:21.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9953648870195326) > weight (0.25) 2021-02-20T11:39:21.114 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:39:21.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:39:21.115 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:39:21.128 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:39:21.116+0000 7f897acd9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 50528) UID: 1000 2021-02-20T11:39:21.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:39:25.331 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:39:25.332 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:39:25.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:39:25.332 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:39:25.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:39:25.333 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:39:25.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:39:25.336 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T11:39:26.116 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:39:26.116 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:39:26.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:39:26.117 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:39:26.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:39:26.117 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:39:26.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:39:26.120 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T11:40:26.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.926076679976197) > weight (0.25) 2021-02-20T11:40:26.122 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:40:26.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:40:26.123 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:40:26.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:40:26.115+0000 7f162cfe4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 49854) UID: 1000 2021-02-20T11:40:26.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:40:35.124 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:40:35.125 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:40:35.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:40:35.126 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:40:35.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:40:35.126 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:40:35.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:40:35.129 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T11:41:08.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26918011896094285) > weight (0.25) 2021-02-20T11:41:08.338 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:41:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:41:08.339 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:41:08.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:41:08.330+0000 7fb12ed26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50722) UID: 1000 2021-02-20T11:41:08.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:41:09.340 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:41:09.341 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:41:09.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:41:09.342 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:41:09.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:41:09.342 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:41:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:41:09.345 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T11:42:04.130 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:42:04.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:42:04.131 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:42:04.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:42:04.132 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:42:04.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:42:04.125+0000 7fe0b8c63700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50823) UID: 1000 2021-02-20T11:42:04.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:42:10.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:42:10.394 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:42:10.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:42:10.402 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:42:10.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:42:10.422 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T11:42:33.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6090199350712255) > weight (0.25) 2021-02-20T11:42:33.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6561714509491181) > weight (0.25) 2021-02-20T11:42:33.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.837440685031302) > weight (0.25) 2021-02-20T11:42:33.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6394687525282475) > weight (0.25) 2021-02-20T11:42:33.348 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T11:43:59.424 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2848728857192647) > weight (0.25) 2021-02-20T11:43:59.425 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T11:43:59.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T11:43:59.426 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:43:59.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T11:43:59.418+0000 7f9b73027700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51204) UID: 1000 2021-02-20T11:43:59.828 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:44:02.427 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T11:44:02.428 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:44:02.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:44:02.429 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T11:44:02.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T11:44:02.429 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T11:44:02.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T11:44:02.432 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T11:44:05.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4784162063292009) > weight (0.25) 2021-02-20T11:44:05.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9836708639651593) > weight (0.25) 2021-02-20T11:44:05.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3760477002842929) > weight (0.25) 2021-02-20T11:44:05.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6343695435172845) > weight (0.25) 2021-02-20T11:44:05.349 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T11:45:48.433 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9625506900500905) > weight (0.25) 2021-02-20T11:45:48.434 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40656558736277215) > weight (0.25) 2021-02-20T11:45:48.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8053127756181471) > weight (0.25) 2021-02-20T11:45:48.435 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:45:48.435 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:45:48.436 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T11:45:48.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:45:48.428+0000 7f892e415700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51104) UID: 1000 2021-02-20T11:45:48.837 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:45:52.350 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3101671404708737) > weight (0.25) 2021-02-20T11:45:52.351 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T11:45:52.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T11:45:52.351 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:45:52.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T11:45:52.344+0000 7fb535434700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51296) UID: 1000 2021-02-20T11:45:52.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:45:53.437 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:45:53.438 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:45:53.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:45:53.439 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:45:53.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:45:53.440 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:45:53.443 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:45:53.443 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T11:45:58.352 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T11:45:58.353 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:45:58.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:45:58.353 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T11:45:58.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T11:45:58.354 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T11:45:58.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T11:45:58.357 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T11:46:23.722 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:46:23.711+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f027400 2021-02-20T11:46:58.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7510266916489083) > weight (0.25) 2021-02-20T11:46:58.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5631243084648141) > weight (0.25) 2021-02-20T11:46:58.359 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:46:58.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:46:58.360 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:46:58.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:46:58.354+0000 7f8b90ce9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49945) UID: 1000 2021-02-20T11:46:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:47:01.361 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:47:01.362 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:47:01.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:47:01.363 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:47:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:47:01.364 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:47:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:47:01.368 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T11:47:10.445 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:47:10.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:47:10.446 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:47:10.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:47:10.438+0000 7f9d9af18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 51396) UID: 1000 2021-02-20T11:47:10.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:47:18.447 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:47:18.448 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:47:18.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:47:18.449 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:47:18.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:47:18.450 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:47:18.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:47:18.460 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T11:48:49.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6164430312001885) > weight (0.25) 2021-02-20T11:48:49.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.930860300974496) > weight (0.25) 2021-02-20T11:48:49.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7549798049699062) > weight (0.25) 2021-02-20T11:48:49.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34597220404556217) > weight (0.25) 2021-02-20T11:48:49.463 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T11:48:52.369 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4866525841402539) > weight (0.25) 2021-02-20T11:48:52.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3809624055355595) > weight (0.25) 2021-02-20T11:48:52.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.27076732338222054) > weight (0.25) 2021-02-20T11:48:52.370 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8090098748716209) > weight (0.25) 2021-02-20T11:48:52.370 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T11:49:57.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9734937079475243) > weight (0.25) 2021-02-20T11:49:57.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6735215799731787) > weight (0.25) 2021-02-20T11:49:57.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27650845234401566) > weight (0.25) 2021-02-20T11:49:57.466 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T11:49:57.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T11:49:57.468 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T11:49:57.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T11:49:57.463+0000 7fab680d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51583) UID: 1000 2021-02-20T11:49:57.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:50:06.468 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T11:50:06.469 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:50:06.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:50:06.470 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T11:50:06.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T11:50:06.471 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T11:50:06.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T11:50:06.474 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T11:50:26.372 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:50:26.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:50:26.373 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:50:26.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:50:26.366+0000 7fe9e5a6c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51013) UID: 1000 2021-02-20T11:50:26.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:50:27.374 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:50:27.374 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:50:27.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:50:27.375 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:50:27.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:50:27.376 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:50:27.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:50:27.378 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T11:51:25.299 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:51:25.288+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e222f0 2021-02-20T11:51:40.379 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5208304618082088) > weight (0.25) 2021-02-20T11:51:40.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2648824344935281) > weight (0.25) 2021-02-20T11:51:40.380 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:51:40.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:51:40.381 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:51:40.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:51:40.372+0000 7fcc80db8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51774) UID: 1000 2021-02-20T11:51:40.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:51:41.382 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:51:41.382 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:51:41.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:51:41.383 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:51:41.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:51:41.384 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:51:41.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:51:41.388 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T11:51:50.475 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6997231478673931) > weight (0.25) 2021-02-20T11:51:50.476 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5420906766758564) > weight (0.25) 2021-02-20T11:51:50.476 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:51:50.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:51:50.477 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:51:50.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T11:51:50.468+0000 7f8700db8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 49469) UID: 1000 2021-02-20T11:51:50.878 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:51:56.477 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:51:56.478 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:51:56.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:51:56.479 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:51:56.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:51:56.479 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:51:56.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:51:56.483 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T11:52:36.743 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T11:52:36.734+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750b1e0 2021-02-20T11:53:38.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6761568408084409) > weight (0.25) 2021-02-20T11:53:38.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4697706748100088) > weight (0.25) 2021-02-20T11:53:38.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6880641283586776) > weight (0.25) 2021-02-20T11:53:38.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3949831639880629) > weight (0.25) 2021-02-20T11:53:38.391 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T11:53:45.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4700538866757443) > weight (0.25) 2021-02-20T11:53:45.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9428209070760162) > weight (0.25) 2021-02-20T11:53:45.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5855864931271069) > weight (0.25) 2021-02-20T11:53:45.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8181260475116984) > weight (0.25) 2021-02-20T11:53:45.489 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T11:55:18.392 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3367109452850503) > weight (0.25) 2021-02-20T11:55:18.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8707435712316219) > weight (0.25) 2021-02-20T11:55:18.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7508156947040194) > weight (0.25) 2021-02-20T11:55:18.393 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:55:18.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:55:18.395 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T11:55:18.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T11:55:18.388+0000 7f2bffbfd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 50429) UID: 1000 2021-02-20T11:55:18.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:55:19.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f1e00 [3/081dcf2b-6aec-4ae9-b0f2-90a8a2eab556] start: already running 2021-02-20T11:55:19.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f1400 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T11:55:19.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f1680 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T11:55:19.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f1900 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T11:55:19.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f2080 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T11:55:19.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e80528700 -1 rbd::mirror::ImageReplayer: 0x55961d3f1b80 [3/3610e9ef-89dd-42ef-8d69-0d79bfec3e56] start: already running 2021-02-20T11:55:19.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x5596214d8a00 [3/674f23ca-7630-45f2-9a82-095e98b79c76] start: already running 2021-02-20T11:55:19.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e7fd27700 -1 rbd::mirror::ImageReplayer: 0x55961d3f2300 [3/828e1151-fa83-405f-a6cf-ec7c06ae0dee] start: already running 2021-02-20T11:55:19.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e80528700 -1 rbd::mirror::ImageReplayer: 0x5596214d9680 [3/a2357b67-5e9d-4bab-b496-4a5956203c0c] start: already running 2021-02-20T11:55:19.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f9e80528700 -1 rbd::mirror::ImageReplayer: 0x55961d3f2580 [3/e2100c63-dab1-4809-b5df-77d32cf43dd0] start: already running 2021-02-20T11:55:19.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972497e00 [3/3a7a8fa6-6a87-4c14-b1a5-91640b3311d6] start: already running 2021-02-20T11:55:19.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972498a80 [3/3ca6eed3-b4cc-42a3-8669-fd637c7a8d69] start: already running 2021-02-20T11:55:19.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972497400 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T11:55:19.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972498f80 [3/4c4f4929-06c7-4203-81c6-a0c9560ab291] start: already running 2021-02-20T11:55:19.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972499200 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T11:55:19.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972499480 [3/b5d77140-8b49-4567-b4bc-f2d77cea0806] start: already running 2021-02-20T11:55:19.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972496c80 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T11:55:19.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972496f00 [3/d83b5ea1-0f06-4764-9c22-8791c64dfe81] start: already running 2021-02-20T11:55:19.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972497180 [3/d9d942d6-60aa-4cd8-a77e-61392adbcc0a] start: already running 2021-02-20T11:55:19.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:55:19.676+0000 7f5ff6020700 -1 rbd::mirror::ImageReplayer: 0x564972499700 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T11:55:21.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:55:21.397 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:55:21.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:55:21.398 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:55:21.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:55:21.398 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:55:21.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:55:21.402 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T11:55:21.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4036335371204628) > weight (0.25) 2021-02-20T11:55:21.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7286367120394701) > weight (0.25) 2021-02-20T11:55:21.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27608739347257216) > weight (0.25) 2021-02-20T11:55:21.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7172962652271994) > weight (0.25) 2021-02-20T11:55:21.491 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T11:56:33.403 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8215533189787769) > weight (0.25) 2021-02-20T11:56:33.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8332743787383726) > weight (0.25) 2021-02-20T11:56:33.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4424559104240464) > weight (0.25) 2021-02-20T11:56:33.404 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T11:56:33.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T11:56:33.405 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T11:56:33.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T11:56:33.398+0000 7f1f9b9be700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 52354) UID: 1000 2021-02-20T11:56:33.492 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T11:56:33.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T11:56:33.493 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T11:56:33.493 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T11:56:33.486+0000 7f4a2daa2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 51873) UID: 1000 2021-02-20T11:56:33.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:56:33.894 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:56:35.406 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T11:56:35.406 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:56:35.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:56:35.407 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T11:56:35.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T11:56:35.408 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T11:56:35.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T11:56:35.410 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T11:56:41.493 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T11:56:41.494 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:56:41.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:56:41.495 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T11:56:41.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T11:56:41.495 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T11:56:41.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T11:56:41.500 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T11:58:06.411 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6532442264997579) > weight (0.25) 2021-02-20T11:58:06.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7862867988649364) > weight (0.25) 2021-02-20T11:58:06.412 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T11:58:06.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T11:58:06.413 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T11:58:06.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T11:58:06.404+0000 7f9e80d29700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 52159) UID: 1000 2021-02-20T11:58:06.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:58:07.414 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T11:58:07.415 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:58:07.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:58:07.415 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T11:58:07.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T11:58:07.416 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T11:58:07.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T11:58:07.419 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T11:58:28.501 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33919673114958016) > weight (0.25) 2021-02-20T11:58:28.501 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4573700436549265) > weight (0.25) 2021-02-20T11:58:28.502 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T11:58:28.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T11:58:28.503 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T11:58:28.503 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T11:58:28.492+0000 7f13b9547700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52258) UID: 1000 2021-02-20T11:58:28.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:58:34.504 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T11:58:34.504 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:58:34.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:58:34.505 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T11:58:34.505 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T11:58:34.506 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T11:58:34.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T11:58:34.509 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T11:59:17.420 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T11:59:17.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T11:59:17.421 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T11:59:17.422 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T11:59:17.422 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T11:59:17.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T11:59:17.414+0000 7f5ff6821700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52059) UID: 1000 2021-02-20T11:59:17.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T11:59:23.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T11:59:23.524 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T11:59:23.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T11:59:23.525 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T11:59:23.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T11:59:23.528 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T11:59:41.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29628836669360614) > weight (0.25) 2021-02-20T11:59:41.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.40971943608942585) > weight (0.25) 2021-02-20T11:59:41.510 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6929362923310048) > weight (0.25) 2021-02-20T11:59:41.511 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.276975973489345) > weight (0.25) 2021-02-20T11:59:41.511 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T12:00:04.768 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:00:04.756+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e20410 2021-02-20T12:00:43.529 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:00:43.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:00:43.532 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:00:43.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:00:43.521+0000 7fd3073e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 52836) UID: 1000 2021-02-20T12:00:43.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:00:44.512 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6846877823748081) > weight (0.25) 2021-02-20T12:00:44.512 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5559749836531754) > weight (0.25) 2021-02-20T12:00:44.513 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:00:44.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:00:44.513 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:00:44.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:00:44.505+0000 7f5d7c397700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52744) UID: 1000 2021-02-20T12:00:44.915 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:00:46.514 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:00:46.515 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:00:46.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:00:46.515 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:00:46.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:00:46.516 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:00:46.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:00:46.519 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:00:49.533 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:00:49.533 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:00:49.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:00:49.534 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:00:49.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:00:49.535 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:00:49.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:00:49.537 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T12:02:19.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7918818526782825) > weight (0.25) 2021-02-20T12:02:19.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7652189903320857) > weight (0.25) 2021-02-20T12:02:19.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9122942368894004) > weight (0.25) 2021-02-20T12:02:19.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4164457822881876) > weight (0.25) 2021-02-20T12:02:19.523 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T12:02:23.538 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2673388606340198) > weight (0.25) 2021-02-20T12:02:23.538 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9891734014558644) > weight (0.25) 2021-02-20T12:02:23.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.35554294161185074) > weight (0.25) 2021-02-20T12:02:23.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6760933287181216) > weight (0.25) 2021-02-20T12:02:23.539 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T12:03:40.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9488817328351661) > weight (0.25) 2021-02-20T12:03:40.541 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:03:40.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:03:40.542 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:03:40.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:03:40.533+0000 7f2a47e78700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51674) UID: 1000 2021-02-20T12:03:40.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:03:48.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:03:48.543 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:03:48.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:03:48.544 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:03:48.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:03:48.545 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:03:48.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:03:48.548 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T12:04:09.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5412343819518622) > weight (0.25) 2021-02-20T12:04:09.524 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:04:09.524 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:04:09.525 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:04:09.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:04:09.516+0000 7fb34074b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51489) UID: 1000 2021-02-20T12:04:09.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:04:13.525 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:04:13.526 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:04:13.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:04:13.527 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:04:13.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:04:13.527 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:04:13.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:04:13.530 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T12:05:11.828 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:05:11.817+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b1eb60 2021-02-20T12:05:31.549 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42576620425781275) > weight (0.25) 2021-02-20T12:05:31.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9369605981372642) > weight (0.25) 2021-02-20T12:05:31.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5457174050419719) > weight (0.25) 2021-02-20T12:05:31.550 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:05:31.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:05:31.551 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:05:31.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:05:31.546+0000 7f7ea639c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 52453) UID: 1000 2021-02-20T12:05:31.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:05:39.552 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:05:39.553 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:05:39.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:05:39.553 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:05:39.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:05:39.554 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:05:39.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:05:39.557 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T12:05:57.532 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:05:57.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:05:57.533 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:05:57.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:05:57.526+0000 7f3da55c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 52544) UID: 1000 2021-02-20T12:05:57.935 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:06:04.533 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:06:04.533 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:06:04.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:06:04.534 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:06:04.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:06:04.535 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:06:04.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:06:04.538 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T12:06:40.558 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:06:40.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:06:40.560 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:06:40.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:06:40.553+0000 7f7c2d9b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53027) UID: 1000 2021-02-20T12:06:40.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:06:41.560 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:06:41.560 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:06:41.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:06:41.561 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:06:41.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:06:41.562 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:06:41.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:06:41.565 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:06:41.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:06:41.809+0000 7f4286e0a700 -1 rbd::mirror::ImageReplayer: 0x564568e63e00 [3/b7d39708-c03f-469a-a6fa-228e371ae7a3] start: already running 2021-02-20T12:06:41.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:06:41.809+0000 7f4286e0a700 -1 rbd::mirror::ImageReplayer: 0x56456d004c80 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T12:07:42.540 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:07:42.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:07:42.542 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:07:42.543 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:07:42.536+0000 7f16ec9dd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53421) UID: 1000 2021-02-20T12:07:42.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:07:50.542 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:07:50.543 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:07:50.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:07:50.543 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:07:50.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:07:50.544 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:07:50.547 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:07:50.547 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T12:08:37.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9132424991044592) > weight (0.25) 2021-02-20T12:08:37.566 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:08:37.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:08:37.568 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:08:37.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:08:37.562+0000 7f4287e0c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53127) UID: 1000 2021-02-20T12:08:37.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:08:43.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:08:43.569 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:08:43.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:08:43.570 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:08:43.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:08:43.571 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:08:43.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:08:43.573 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T12:09:42.548 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7382772777990557) > weight (0.25) 2021-02-20T12:09:42.548 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.31392677165474847) > weight (0.25) 2021-02-20T12:09:42.549 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:09:42.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:09:42.549 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:09:42.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:09:42.545+0000 7f9d86e1c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52936) UID: 1000 2021-02-20T12:09:42.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:09:44.550 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:09:44.550 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:09:44.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:09:44.551 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:09:44.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:09:44.552 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:09:44.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:09:44.555 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T12:10:01.575 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:10:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:10:01.576 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:10:01.576 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:10:01.576 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:10:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:10:01.573+0000 7faa029b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53512) UID: 1000 2021-02-20T12:10:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:10:07.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:10:07.678 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:10:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:10:07.679 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:10:07.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:10:07.682 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T12:10:52.556 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:10:52.558 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:10:52.558 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:10:52.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:10:52.551+0000 7f8455d94700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53611) UID: 1000 2021-02-20T12:10:52.960 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:10:53.559 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:10:53.560 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:10:53.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:10:53.561 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:10:53.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:10:53.561 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:10:53.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:10:53.565 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T12:11:24.898 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:11:24.888+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750b1e0 2021-02-20T12:11:28.683 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.960977746657744) > weight (0.25) 2021-02-20T12:11:28.683 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6474391619138037) > weight (0.25) 2021-02-20T12:11:28.683 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:11:28.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:11:28.684 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:11:28.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:11:28.679+0000 7f37c5bab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 52644) UID: 1000 2021-02-20T12:11:29.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:11:37.686 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:11:37.686 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:11:37.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:11:37.687 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:11:37.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:11:37.687 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:11:37.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:11:37.690 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T12:12:22.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8780035469928935) > weight (0.25) 2021-02-20T12:12:22.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6468228788834129) > weight (0.25) 2021-02-20T12:12:22.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9080059184535113) > weight (0.25) 2021-02-20T12:12:22.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2818924267506616) > weight (0.25) 2021-02-20T12:12:22.567 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T12:12:44.274 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:12:44.269+0000 7f803c4bb700 -1 reset not still connected to 0x55acb039b670 2021-02-20T12:12:44.275 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:12:44.269+0000 7f803c4bb700 -1 reset not still connected to 0x55acd5908750 2021-02-20T12:13:14.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5335766316780208) > weight (0.25) 2021-02-20T12:13:14.692 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9763039343482117) > weight (0.25) 2021-02-20T12:13:14.692 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5660707833011027) > weight (0.25) 2021-02-20T12:13:14.692 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:13:14.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:13:14.693 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:13:14.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:13:14.688+0000 7f1666e9d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 53322) UID: 1000 2021-02-20T12:13:15.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:13:23.694 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:13:23.694 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:13:23.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:13:23.695 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:13:23.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:13:23.696 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:13:23.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:13:23.699 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T12:13:24.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3272035016443541) > weight (0.25) 2021-02-20T12:13:24.568 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:13:24.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:13:24.568 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:13:24.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:13:24.564+0000 7fdc595c8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53227) UID: 1000 2021-02-20T12:13:24.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:13:32.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:13:32.569 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:13:32.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:13:32.570 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:13:32.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:13:32.571 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:13:32.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:13:32.574 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:14:56.575 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4031441310024366) > weight (0.25) 2021-02-20T12:14:56.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.876143390279363) > weight (0.25) 2021-02-20T12:14:56.576 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:14:56.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:14:56.578 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:14:56.578 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:14:56.574+0000 7f979e786700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 53803) UID: 1000 2021-02-20T12:14:56.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:14:57.578 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:14:57.579 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:14:57.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:14:57.579 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:14:57.580 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:14:57.580 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:14:57.583 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:14:57.583 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:15:04.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44299248237754074) > weight (0.25) 2021-02-20T12:15:04.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25890922707358943) > weight (0.25) 2021-02-20T12:15:04.700 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:15:04.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:15:04.701 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:15:04.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:15:04.698+0000 7fdf8c0eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54087) UID: 1000 2021-02-20T12:15:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:15:11.702 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:15:11.703 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:15:11.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:15:11.703 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:15:11.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:15:11.704 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:15:11.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:15:11.707 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T12:15:57.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9124849494900116) > weight (0.25) 2021-02-20T12:15:57.585 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:15:57.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:15:57.586 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:15:57.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:15:57.581+0000 7f72f0755700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54277) UID: 1000 2021-02-20T12:15:57.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:16:00.586 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:16:00.587 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:16:00.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:16:00.587 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:16:00.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:16:00.588 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:16:00.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:16:00.591 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:16:26.708 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:16:26.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:16:26.709 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:16:26.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:16:26.704+0000 7f5f4cbd4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53897) UID: 1000 2021-02-20T12:16:27.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:16:28.711 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:16:28.711 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:16:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:16:28.712 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:16:28.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:16:28.712 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:16:28.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:16:28.715 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:17:10.592 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:17:10.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:17:10.593 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:17:10.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:17:10.591+0000 7fe32c1d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53996) UID: 1000 2021-02-20T12:17:10.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:17:13.594 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:17:13.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:17:13.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:17:13.596 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:17:13.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:17:13.596 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:17:13.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:17:13.599 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:18:24.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6445511127263758) > weight (0.25) 2021-02-20T12:18:24.726 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5017094483607304) > weight (0.25) 2021-02-20T12:18:24.726 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6262350797823105) > weight (0.25) 2021-02-20T12:18:24.727 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8425793015352808) > weight (0.25) 2021-02-20T12:18:24.727 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:19:09.600 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:19:09.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:19:09.601 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:19:09.602 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:19:09.597+0000 7fd626b04700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54766) UID: 1000 2021-02-20T12:19:10.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:19:12.603 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:19:12.603 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:19:12.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:19:12.604 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:19:12.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:19:12.605 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:19:12.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:19:12.608 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T12:19:48.728 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7915670133709516) > weight (0.25) 2021-02-20T12:19:48.728 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:19:48.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:19:48.729 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:19:48.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:19:48.724+0000 7f95e3fc6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 53703) UID: 1000 2021-02-20T12:19:49.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:19:49.730 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:19:49.731 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:19:49.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:19:49.731 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:19:49.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:19:49.732 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:19:49.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:19:49.735 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T12:19:50.123 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:19:50.116+0000 7f508e15e700 -1 reset not still connected to 0x55ac87975330 2021-02-20T12:20:57.841 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:20:57.838+0000 7f508e15e700 -1 reset not still connected to 0x55ac87273ad0 2021-02-20T12:21:00.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46465621524130774) > weight (0.25) 2021-02-20T12:21:00.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5767263404793999) > weight (0.25) 2021-02-20T12:21:00.609 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:21:00.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:21:00.610 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:21:00.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:21:00.606+0000 7fb825940700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 54378) UID: 1000 2021-02-20T12:21:01.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:21:09.611 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:21:09.612 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:21:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:21:09.612 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:21:09.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:21:09.613 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:21:09.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:21:09.616 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:21:37.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5029790551514155) > weight (0.25) 2021-02-20T12:21:37.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7273273836216655) > weight (0.25) 2021-02-20T12:21:37.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8896326974391814) > weight (0.25) 2021-02-20T12:21:37.737 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:21:37.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:21:37.738 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:21:37.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:21:37.733+0000 7f2e63fe2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 54186) UID: 1000 2021-02-20T12:21:38.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:21:46.738 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:21:46.739 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:21:46.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:21:46.740 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:21:46.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:21:46.740 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:21:46.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:21:46.744 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T12:22:06.292 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.289+0000 7f508e15e700 -1 reset not still connected to 0x55ac81674270 2021-02-20T12:22:06.293 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.289+0000 7f508e15e700 -1 reset not still connected to 0x55ac81677400 2021-02-20T12:22:06.296 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.289+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b995f0 2021-02-20T12:22:06.296 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.289+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c9ba0 2021-02-20T12:22:06.296 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.293+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f7450 2021-02-20T12:22:06.297 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:22:06.293+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f8e50 2021-02-20T12:22:19.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7702841204724264) > weight (0.25) 2021-02-20T12:22:19.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7202173255554744) > weight (0.25) 2021-02-20T12:22:19.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9830558087863246) > weight (0.25) 2021-02-20T12:22:19.617 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5241972894099812) > weight (0.25) 2021-02-20T12:22:19.618 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T12:23:04.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5735906501626239) > weight (0.25) 2021-02-20T12:23:04.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8894454273650647) > weight (0.25) 2021-02-20T12:23:04.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7188380004458288) > weight (0.25) 2021-02-20T12:23:04.746 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:23:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:23:04.747 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T12:23:04.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:23:04.748 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:23:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:23:04.744+0000 7f447dc4a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 55142) UID: 1000 2021-02-20T12:23:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:23:10.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:23:10.851 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:23:10.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:23:10.852 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:23:10.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:23:10.859 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T12:24:05.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3499449686373213) > weight (0.25) 2021-02-20T12:24:05.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6756149345566741) > weight (0.25) 2021-02-20T12:24:05.620 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:24:05.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:24:05.621 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:24:05.621 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:24:05.618+0000 7f8d59ddb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 55051) UID: 1000 2021-02-20T12:24:06.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:24:08.621 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:24:08.622 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:24:08.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:24:08.622 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:24:08.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:24:08.623 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:24:08.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:24:08.626 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T12:25:06.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6422685895361616) > weight (0.25) 2021-02-20T12:25:06.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9575262708923024) > weight (0.25) 2021-02-20T12:25:06.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.392673161851255) > weight (0.25) 2021-02-20T12:25:06.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.575829944855654) > weight (0.25) 2021-02-20T12:25:06.861 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T12:25:29.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47511752752471803) > weight (0.25) 2021-02-20T12:25:29.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.42475143791520076) > weight (0.25) 2021-02-20T12:25:29.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2888788449409554) > weight (0.25) 2021-02-20T12:25:29.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.628572329635377) > weight (0.25) 2021-02-20T12:25:29.628 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T12:26:29.629 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:26:29.630 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:26:29.630 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:26:29.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:26:29.627+0000 7fa3bd6e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54858) UID: 1000 2021-02-20T12:26:30.032 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:26:31.631 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:26:31.631 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:26:31.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:26:31.632 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:26:31.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:26:31.633 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:26:31.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:26:31.636 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T12:26:38.883 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:26:38.879+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1335d40 2021-02-20T12:26:38.886 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:26:38.879+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1336080 2021-02-20T12:26:57.862 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:26:57.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:26:57.863 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:26:57.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:26:57.863+0000 7f234cc18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54664) UID: 1000 2021-02-20T12:26:58.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:27:03.864 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:27:03.870 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:27:03.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:27:03.871 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:27:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:27:03.872 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:27:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:27:03.875 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T12:28:11.896 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:28:11.892+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f0f35a0 2021-02-20T12:28:11.897 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:28:11.892+0000 7f508e15e700 -1 reset not still connected to 0x55ac81674270 2021-02-20T12:28:11.901 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:28:11.896+0000 7f508e15e700 -1 reset not still connected to 0x55ac81676630 2021-02-20T12:28:12.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6140852763591742) > weight (0.25) 2021-02-20T12:28:12.637 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:28:12.638 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:28:12.638 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:28:12.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:28:12.637+0000 7f45f6a17700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 54573) UID: 1000 2021-02-20T12:28:13.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:28:13.639 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:28:13.639 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:28:13.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:28:13.640 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:28:13.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:28:13.641 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:28:13.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:28:13.643 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T12:28:15.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2755938316442348) > weight (0.25) 2021-02-20T12:28:15.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.44240871115039715) > weight (0.25) 2021-02-20T12:28:15.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3220110919554082) > weight (0.25) 2021-02-20T12:28:15.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45320729776502944) > weight (0.25) 2021-02-20T12:28:15.877 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:29:15.644 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:29:15.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:29:15.646 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:29:15.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:29:15.644+0000 7f48d7f94700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55437) UID: 1000 2021-02-20T12:29:16.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:29:18.647 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:29:18.647 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:29:18.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:29:18.648 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:29:18.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:29:18.648 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:29:18.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:29:18.651 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T12:29:45.837 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:29:45.834+0000 7f508e15e700 -1 reset not still connected to 0x55ac81675e10 2021-02-20T12:29:48.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26345961978295984) > weight (0.25) 2021-02-20T12:29:48.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8428201480845583) > weight (0.25) 2021-02-20T12:29:48.879 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30043896718998875) > weight (0.25) 2021-02-20T12:29:48.879 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7559151487004091) > weight (0.25) 2021-02-20T12:29:48.879 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T12:30:49.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48793370938203107) > weight (0.25) 2021-02-20T12:30:49.653 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:30:49.654 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:30:49.654 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:30:49.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:30:49.654+0000 7f54a6095700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55627) UID: 1000 2021-02-20T12:30:50.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:30:57.656 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:30:57.656 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:30:57.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:30:57.657 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:30:57.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:30:57.658 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:30:57.661 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:30:57.661 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T12:31:02.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4747160383846858) > weight (0.25) 2021-02-20T12:31:02.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7927186789478868) > weight (0.25) 2021-02-20T12:31:02.881 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:31:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:31:02.882 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:31:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:31:02.881+0000 7f1e340d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54474) UID: 1000 2021-02-20T12:31:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:31:09.882 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:31:09.883 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:31:09.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:31:09.884 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:31:09.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:31:09.884 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:31:09.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:31:09.887 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T12:32:25.662 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:32:25.662 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:32:25.663 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:32:25.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:32:25.663+0000 7f43400ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55718) UID: 1000 2021-02-20T12:32:26.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:32:26.663 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:32:26.664 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:32:26.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:32:26.665 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:32:26.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:32:26.665 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:32:26.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:32:26.668 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T12:32:46.889 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:32:46.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:32:46.890 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:32:46.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:32:46.887+0000 7f02fc182700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55528) UID: 1000 2021-02-20T12:32:47.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:32:51.890 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:32:51.891 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:32:51.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:32:51.892 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:32:51.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:32:51.892 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:32:51.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:32:51.899 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T12:34:21.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.44882994635406526) > weight (0.25) 2021-02-20T12:34:21.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.707635800386672) > weight (0.25) 2021-02-20T12:34:21.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2973015178538355) > weight (0.25) 2021-02-20T12:34:21.670 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:34:21.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:34:21.672 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:34:21.672 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:34:21.669+0000 7f465ab8a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51968) UID: 1000 2021-02-20T12:34:22.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:34:28.672 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:34:28.673 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:34:28.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:34:28.674 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:34:28.674 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:34:28.674 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:34:28.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:34:28.679 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T12:34:28.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.728701368860803) > weight (0.25) 2021-02-20T12:34:28.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8359041377343011) > weight (0.25) 2021-02-20T12:34:28.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.35609193486279833) > weight (0.25) 2021-02-20T12:34:28.901 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:34:28.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:34:28.902 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:34:28.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:34:28.901+0000 7f6e8aee9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 55241) UID: 1000 2021-02-20T12:34:29.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:34:37.903 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:34:37.903 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:34:37.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:34:37.904 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:34:37.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:34:37.905 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:34:37.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:34:37.907 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T12:35:51.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46776258765686474) > weight (0.25) 2021-02-20T12:35:51.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2920436692947326) > weight (0.25) 2021-02-20T12:35:51.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9500978854096899) > weight (0.25) 2021-02-20T12:35:51.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9688486494320503) > weight (0.25) 2021-02-20T12:35:51.681 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T12:36:25.909 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5542618454107998) > weight (0.25) 2021-02-20T12:36:25.909 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:36:25.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:36:25.910 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:36:25.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:36:25.910+0000 7fe853e90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54949) UID: 1000 2021-02-20T12:36:26.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:36:26.911 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:36:26.912 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:36:26.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:36:26.912 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:36:26.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:36:26.913 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:36:26.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:36:26.916 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T12:37:19.682 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:37:19.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:37:19.684 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:37:19.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:37:19.681+0000 7ff4d1965700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56004) UID: 1000 2021-02-20T12:37:20.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:37:22.685 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:37:22.685 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:37:22.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:37:22.686 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:37:22.686 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:37:22.687 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:37:22.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:37:22.690 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T12:37:57.917 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:37:57.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:37:57.918 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:37:57.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:37:57.916+0000 7faf24d5b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56095) UID: 1000 2021-02-20T12:37:58.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:37:59.919 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:37:59.919 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:37:59.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:37:59.920 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:37:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:37:59.921 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:37:59.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:37:59.924 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T12:38:31.410 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:38:31.406+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133c630 2021-02-20T12:38:44.690 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:38:44.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:38:44.691 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:38:44.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:38:44.691+0000 7f8282d84700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56489) UID: 1000 2021-02-20T12:38:45.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:38:47.692 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:38:47.693 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:38:47.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:38:47.694 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:38:47.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:38:47.694 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:38:47.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:38:47.697 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T12:39:16.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3384328711335256) > weight (0.25) 2021-02-20T12:39:16.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8001363143145465) > weight (0.25) 2021-02-20T12:39:16.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8703509522537163) > weight (0.25) 2021-02-20T12:39:16.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2891005027201292) > weight (0.25) 2021-02-20T12:39:16.926 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T12:40:33.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8218300520754201) > weight (0.25) 2021-02-20T12:40:33.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8873190925493329) > weight (0.25) 2021-02-20T12:40:33.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9921314172606457) > weight (0.25) 2021-02-20T12:40:33.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2633257943571331) > weight (0.25) 2021-02-20T12:40:33.699 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T12:41:05.927 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8363167076202572) > weight (0.25) 2021-02-20T12:41:05.928 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3333940139472461) > weight (0.25) 2021-02-20T12:41:05.928 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.735261161350417) > weight (0.25) 2021-02-20T12:41:05.928 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:41:05.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:41:05.929 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:41:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:41:05.928+0000 7f9638666700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 56285) UID: 1000 2021-02-20T12:41:06.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:41:06.930 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:41:06.931 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:41:06.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:41:06.932 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:41:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:41:06.932 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:41:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:41:06.935 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T12:42:08.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42715202013851994) > weight (0.25) 2021-02-20T12:42:08.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7826107794781599) > weight (0.25) 2021-02-20T12:42:08.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6273924707887498) > weight (0.25) 2021-02-20T12:42:08.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7005799037936348) > weight (0.25) 2021-02-20T12:42:08.702 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T12:42:40.108 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:42:40.106+0000 7f803c4bb700 -1 reset not still connected to 0x55acb7d33040 2021-02-20T12:42:40.109 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:42:40.106+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9176cb0 2021-02-20T12:42:40.109 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:42:40.106+0000 7f803c4bb700 -1 reset not still connected to 0x55acba9a1040 2021-02-20T12:42:40.936 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:42:40.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:42:40.937 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:42:40.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:42:40.934+0000 7fae68d84700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56580) UID: 1000 2021-02-20T12:42:41.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:42:49.939 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:42:49.939 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:42:49.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:42:49.940 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:42:49.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:42:49.940 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:42:49.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:42:49.943 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T12:43:17.703 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30031724036331287) > weight (0.25) 2021-02-20T12:43:17.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2828575663644015) > weight (0.25) 2021-02-20T12:43:17.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26499939327693534) > weight (0.25) 2021-02-20T12:43:17.704 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:43:17.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:43:17.705 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:43:17.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:43:17.705+0000 7f5aca1a0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 56194) UID: 1000 2021-02-20T12:43:18.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:43:20.706 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:43:20.706 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:43:20.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:43:20.707 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:43:20.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:43:20.707 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:43:20.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:43:20.711 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T12:44:28.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48104696327475205) > weight (0.25) 2021-02-20T12:44:28.945 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4966535346657105) > weight (0.25) 2021-02-20T12:44:28.945 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:44:28.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:44:28.946 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:44:28.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:44:28.944+0000 7f147c92c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 55904) UID: 1000 2021-02-20T12:44:29.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:44:35.949 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:44:35.949 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:44:35.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:44:35.950 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:44:35.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:44:35.951 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:44:35.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:44:35.955 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T12:45:09.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3662012468764164) > weight (0.25) 2021-02-20T12:45:09.713 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:45:09.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:45:09.714 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:45:09.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:45:09.711+0000 7faab28d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 55812) UID: 1000 2021-02-20T12:45:10.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:45:12.715 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:45:12.715 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:45:12.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:45:12.716 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:45:12.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:45:12.716 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:45:12.719 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:45:12.719 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T12:46:01.178 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T12:46:01.174+0000 7fc273bc1700 -1 reset not still connected to 0x5611816aad80 2021-02-20T12:46:02.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6032807667315716) > weight (0.25) 2021-02-20T12:46:02.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8577965945901393) > weight (0.25) 2021-02-20T12:46:02.956 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:46:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:46:02.957 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:46:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:46:02.954+0000 7fde596b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57131) UID: 1000 2021-02-20T12:46:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:46:08.958 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:46:08.958 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:46:08.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:46:08.959 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:46:08.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:46:08.959 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:46:08.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:46:08.962 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T12:46:12.520 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T12:46:12.515+0000 7f62c7ffc700 -1 reset not still connected to 0x555b4c51c750 2021-02-20T12:46:12.520 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T12:46:12.519+0000 7f62c7ffc700 -1 reset not still connected to 0x555b4c51fcf0 2021-02-20T12:46:40.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.325863812568709) > weight (0.25) 2021-02-20T12:46:40.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5178834369036232) > weight (0.25) 2021-02-20T12:46:40.721 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:46:40.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:46:40.721 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:46:40.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:46:40.717+0000 7f2407b93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 55340) UID: 1000 2021-02-20T12:46:41.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:46:41.723 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:46:41.723 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:46:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:46:41.724 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:46:41.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:46:41.725 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:46:41.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:46:41.728 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:47:32.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5818585168850825) > weight (0.25) 2021-02-20T12:47:32.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3763824706080531) > weight (0.25) 2021-02-20T12:47:32.964 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:47:32.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:47:32.965 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:47:32.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:47:32.960+0000 7f66454ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57341) UID: 1000 2021-02-20T12:47:33.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:47:37.966 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:47:37.967 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:47:37.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:47:37.967 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:47:37.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:47:37.968 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:47:37.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:47:37.971 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T12:47:51.728 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6754222161691006) > weight (0.25) 2021-02-20T12:47:51.729 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:47:51.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:47:51.730 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:47:51.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:47:51.723+0000 7fcbba889700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 57250) UID: 1000 2021-02-20T12:47:52.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:48:00.730 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:48:00.731 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:48:00.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:48:00.732 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:48:00.732 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:48:00.732 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:48:00.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:48:00.735 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T12:48:42.012 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T12:48:42.010+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9175930 2021-02-20T12:49:24.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6735870828298813) > weight (0.25) 2021-02-20T12:49:24.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7329973694981012) > weight (0.25) 2021-02-20T12:49:24.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9411544365690588) > weight (0.25) 2021-02-20T12:49:24.973 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T12:49:24.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T12:49:24.973 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:49:24.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T12:49:24.969+0000 7fc90cc24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 56773) UID: 1000 2021-02-20T12:49:25.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:49:29.975 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T12:49:29.976 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:49:29.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:49:29.976 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T12:49:29.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T12:49:29.977 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T12:49:29.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T12:49:29.980 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T12:49:49.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8997005235186318) > weight (0.25) 2021-02-20T12:49:49.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8686172015813498) > weight (0.25) 2021-02-20T12:49:49.736 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:49:49.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:49:49.737 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T12:49:49.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:49:49.733+0000 7fd5653ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57441) UID: 1000 2021-02-20T12:49:50.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:49:50.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:49:50.965+0000 7fe418c99700 -1 rbd::mirror::ImageReplayer: 0x55cf38616580 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T12:49:56.739 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:49:56.739 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:49:56.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:49:56.740 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:49:56.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:49:56.740 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:49:56.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:49:56.750 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T12:51:07.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5023507247155813) > weight (0.25) 2021-02-20T12:51:07.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7790062451512021) > weight (0.25) 2021-02-20T12:51:07.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6778471558029844) > weight (0.25) 2021-02-20T12:51:07.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5102903167867203) > weight (0.25) 2021-02-20T12:51:07.983 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T12:51:32.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4786607400106385) > weight (0.25) 2021-02-20T12:51:32.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8264609142633089) > weight (0.25) 2021-02-20T12:51:32.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2740554472154867) > weight (0.25) 2021-02-20T12:51:32.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7764898148624538) > weight (0.25) 2021-02-20T12:51:32.754 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T12:52:23.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3467855610268513) > weight (0.25) 2021-02-20T12:52:23.984 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9939403208441941) > weight (0.25) 2021-02-20T12:52:23.985 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:52:23.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:52:23.987 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T12:52:23.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:52:23.982+0000 7fc97008c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57533) UID: 1000 2021-02-20T12:52:24.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:52:28.988 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:52:28.988 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:52:28.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:52:28.989 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:52:28.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:52:28.989 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:52:28.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:52:28.993 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T12:53:21.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7640707542643355) > weight (0.25) 2021-02-20T12:53:21.762 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.706129833073071) > weight (0.25) 2021-02-20T12:53:21.762 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4206184613041454) > weight (0.25) 2021-02-20T12:53:21.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7905419827428753) > weight (0.25) 2021-02-20T12:53:21.763 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T12:53:55.993 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:53:55.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:53:55.995 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T12:53:55.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:53:55.992+0000 7f9151392700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56873) UID: 1000 2021-02-20T12:53:56.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:54:00.978 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T12:54:00.971+0000 7fc273bc1700 -1 reset not still connected to 0x56119260ac30 2021-02-20T12:54:04.996 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:54:04.996 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:54:04.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:54:04.997 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:54:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:54:04.998 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:54:05.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:54:05.001 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T12:54:31.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3287643894803771) > weight (0.25) 2021-02-20T12:54:31.770 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T12:54:31.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T12:54:31.771 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T12:54:31.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T12:54:31.771+0000 7ff10f9a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 57633) UID: 1000 2021-02-20T12:54:32.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:54:35.772 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T12:54:35.778 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:54:35.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:54:35.779 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T12:54:35.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T12:54:35.780 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T12:54:35.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T12:54:35.783 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T12:55:07.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5347339614290417) > weight (0.25) 2021-02-20T12:55:07.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3781724701054411) > weight (0.25) 2021-02-20T12:55:07.005 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T12:55:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T12:55:07.006 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:55:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T12:55:07.002+0000 7f9731e7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 57917) UID: 1000 2021-02-20T12:55:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:55:15.007 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T12:55:15.007 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:55:15.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:55:15.008 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T12:55:15.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T12:55:15.009 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T12:55:15.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T12:55:15.012 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T12:55:18.805 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:55:18.802+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f99b0 2021-02-20T12:55:18.805 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T12:55:18.802+0000 7f508e15e700 -1 reset not still connected to 0x55ac9133add0 2021-02-20T12:56:18.785 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8153449917651083) > weight (0.25) 2021-02-20T12:56:18.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28937430368518346) > weight (0.25) 2021-02-20T12:56:18.786 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T12:56:18.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T12:56:18.787 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T12:56:18.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T12:56:18.784+0000 7feb8c135700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57825) UID: 1000 2021-02-20T12:56:19.188 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:56:20.788 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T12:56:20.788 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:56:20.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:56:20.789 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T12:56:20.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T12:56:20.790 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T12:56:20.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T12:56:20.793 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T12:56:49.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49497335963216094) > weight (0.25) 2021-02-20T12:56:49.013 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T12:56:49.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T12:56:49.014 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T12:56:49.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T12:56:49.008+0000 7fec39080700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 56388) UID: 1000 2021-02-20T12:56:49.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:56:52.015 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T12:56:52.015 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:56:52.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:56:52.016 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T12:56:52.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T12:56:52.017 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T12:56:52.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T12:56:52.027 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T12:57:24.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2562584235966313) > weight (0.25) 2021-02-20T12:57:24.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9630335123480702) > weight (0.25) 2021-02-20T12:57:24.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7905334996798495) > weight (0.25) 2021-02-20T12:57:24.795 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T12:57:24.796 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T12:57:24.796 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T12:57:24.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:57:24.795+0000 7fe41949a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 56973) UID: 1000 2021-02-20T12:57:25.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:57:30.797 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T12:57:30.805 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:57:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:57:30.806 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T12:57:30.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T12:57:30.807 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T12:57:30.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T12:57:30.810 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T12:58:10.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9148325759449316) > weight (0.25) 2021-02-20T12:58:10.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5524374633133448) > weight (0.25) 2021-02-20T12:58:10.044 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5277756765193541) > weight (0.25) 2021-02-20T12:58:10.044 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7284591765056492) > weight (0.25) 2021-02-20T12:58:10.045 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T12:59:19.046 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T12:59:19.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T12:59:19.056 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T12:59:19.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T12:59:19.053+0000 7fc1e0163700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58019) UID: 1000 2021-02-20T12:59:19.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:59:20.056 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T12:59:20.057 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:59:20.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:59:20.057 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T12:59:20.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T12:59:20.058 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T12:59:20.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T12:59:20.065 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T12:59:29.811 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T12:59:29.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T12:59:29.812 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T12:59:29.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T12:59:29.808+0000 7feeac980700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56679) UID: 1000 2021-02-20T12:59:30.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T12:59:31.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.096+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebde00 [3/a2357b67-5e9d-4bab-b496-4a5956203c0c] start: already running 2021-02-20T12:59:31.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.096+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebe080 [3/b7d39708-c03f-469a-a6fa-228e371ae7a3] start: already running 2021-02-20T12:59:31.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.100+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebed00 [3/c21d02f7-4a75-40cc-89e6-32ad02b9d7d4] start: already running 2021-02-20T12:59:31.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.100+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebd400 [3/d9d942d6-60aa-4cd8-a77e-61392adbcc0a] start: already running 2021-02-20T12:59:31.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.100+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebd680 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T12:59:31.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T12:59:31.100+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615f6ebd900 [3/e2100c63-dab1-4809-b5df-77d32cf43dd0] start: already running 2021-02-20T12:59:37.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T12:59:37.813 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T12:59:37.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T12:59:37.814 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T12:59:37.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T12:59:37.815 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T12:59:37.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T12:59:37.822 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T13:00:46.823 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9398773639665525) > weight (0.25) 2021-02-20T13:00:46.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7644990041115495) > weight (0.25) 2021-02-20T13:00:46.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6477227197670797) > weight (0.25) 2021-02-20T13:00:46.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6054066184304916) > weight (0.25) 2021-02-20T13:00:46.834 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T13:00:53.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.38363740166140836) > weight (0.25) 2021-02-20T13:00:53.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2696468169536803) > weight (0.25) 2021-02-20T13:00:53.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7834104435429208) > weight (0.25) 2021-02-20T13:00:53.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2951830182051828) > weight (0.25) 2021-02-20T13:00:53.082 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T13:02:25.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.940810967156255) > weight (0.25) 2021-02-20T13:02:25.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49951458028265705) > weight (0.25) 2021-02-20T13:02:25.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7616234423277436) > weight (0.25) 2021-02-20T13:02:25.867 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8257956048115062) > weight (0.25) 2021-02-20T13:02:25.867 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T13:02:30.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7792642212103718) > weight (0.25) 2021-02-20T13:02:30.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9539126877628142) > weight (0.25) 2021-02-20T13:02:30.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5389974006730983) > weight (0.25) 2021-02-20T13:02:30.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6516178533373923) > weight (0.25) 2021-02-20T13:02:30.092 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T13:03:52.093 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:03:52.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:03:52.103 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:03:52.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:03:52.103+0000 7fdd2a563700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58595) UID: 1000 2021-02-20T13:03:52.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:03:54.104 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:03:54.108 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:03:54.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:03:54.109 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:03:54.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:03:54.110 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:03:54.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:03:54.113 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T13:04:17.869 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:04:17.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:04:17.878 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:04:17.878 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:04:17.878 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:04:17.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:04:17.874+0000 7f322b9e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58694) UID: 1000 2021-02-20T13:04:18.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:04:23.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:04:23.990 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:04:23.990 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:04:23.991 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:04:23.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:04:23.994 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T13:05:24.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7581467496928315) > weight (0.25) 2021-02-20T13:05:24.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8803382027740433) > weight (0.25) 2021-02-20T13:05:24.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9474371025784727) > weight (0.25) 2021-02-20T13:05:24.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5229167597355604) > weight (0.25) 2021-02-20T13:05:24.157 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T13:05:31.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7725660582756306) > weight (0.25) 2021-02-20T13:05:32.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9376323063654244) > weight (0.25) 2021-02-20T13:05:32.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7263644678012812) > weight (0.25) 2021-02-20T13:05:32.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2876597386831715) > weight (0.25) 2021-02-20T13:05:32.003 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:06:57.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4854965456995962) > weight (0.25) 2021-02-20T13:06:57.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2590805408873511) > weight (0.25) 2021-02-20T13:06:57.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4172676047223368) > weight (0.25) 2021-02-20T13:06:57.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7512996076447058) > weight (0.25) 2021-02-20T13:06:57.008 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T13:07:03.158 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:07:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:07:03.165 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:07:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:07:03.163+0000 7f68e6a7a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58788) UID: 1000 2021-02-20T13:07:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:07:10.166 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:07:10.166 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:07:10.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:07:10.167 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:07:10.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:07:10.168 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:07:10.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:07:10.171 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T13:07:47.704 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T13:07:47.698+0000 7fc273bc1700 -1 reset not still connected to 0x56118dcbc1a0 2021-02-20T13:07:47.730 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T13:07:47.702+0000 7fc273bc1700 -1 reset not still connected to 0x56118dcbdad0 2021-02-20T13:08:31.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6891948475207111) > weight (0.25) 2021-02-20T13:08:31.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5897502570624781) > weight (0.25) 2021-02-20T13:08:31.019 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:08:31.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:08:31.021 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:08:31.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:08:31.017+0000 7f3ba9106700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 58311) UID: 1000 2021-02-20T13:08:31.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:08:37.022 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:08:37.030 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:08:37.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:08:37.031 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:08:37.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:08:37.032 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:08:37.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:08:37.035 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T13:08:42.633 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T13:08:42.628+0000 7f803c4bb700 -1 reset not still connected to 0x55acc96970c0 2021-02-20T13:09:04.172 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:09:04.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:09:04.180 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:09:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:09:04.176+0000 7f0c24cae700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58983) UID: 1000 2021-02-20T13:09:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:09:11.182 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:09:11.189 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:09:11.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:09:11.190 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:09:11.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:09:11.191 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:09:11.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:09:11.195 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T13:10:31.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42077139738428804) > weight (0.25) 2021-02-20T13:10:31.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3667712436519541) > weight (0.25) 2021-02-20T13:10:31.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8834475612605464) > weight (0.25) 2021-02-20T13:10:31.049 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2642442999984058) > weight (0.25) 2021-02-20T13:10:31.049 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T13:10:48.197 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8455623403280639) > weight (0.25) 2021-02-20T13:10:48.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3932423419979659) > weight (0.25) 2021-02-20T13:10:48.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5804732328869557) > weight (0.25) 2021-02-20T13:10:48.215 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:10:48.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:10:48.216 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:10:48.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T13:10:48.214+0000 7f58e8143700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 57726) UID: 1000 2021-02-20T13:10:48.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:10:49.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.466+0000 7fbd01f57700 -1 rbd::mirror::ImageReplayer: 0x564ba308c280 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T13:10:49.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564b9f036c80 [3/3610e9ef-89dd-42ef-8d69-0d79bfec3e56] start: already running 2021-02-20T13:10:49.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564b9f037180 [3/6a6f5866-70c0-4063-8b45-7c49d8fcdf4b] start: already running 2021-02-20T13:10:49.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564ba308cc80 [3/75792560-55ec-454a-bb3c-f7a23e6221d8] start: already running 2021-02-20T13:10:49.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564b9f039980 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T13:10:49.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564b9f036f00 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T13:10:49.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564b9f039700 [3/90d8a604-9cc2-4227-8cf3-7e7ff6b73bd0] start: already running 2021-02-20T13:10:49.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01f57700 -1 rbd::mirror::ImageReplayer: 0x564b9f039c00 [3/9a469893-68d6-4d91-a371-3899fcc47454] start: already running 2021-02-20T13:10:49.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:10:49.474+0000 7fbd01756700 -1 rbd::mirror::ImageReplayer: 0x564ba308c780 [3/d9d942d6-60aa-4cd8-a77e-61392adbcc0a] start: already running 2021-02-20T13:10:50.031 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T13:10:49.904+0000 7f62c7ffc700 -1 reset not still connected to 0x555b4f64c000 2021-02-20T13:10:50.031 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T13:10:49.904+0000 7f62c7ffc700 -1 reset not still connected to 0x555b53af9790 2021-02-20T13:10:50.032 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T13:10:49.904+0000 7f62c7ffc700 -1 reset not still connected to 0x555b53afb260 2021-02-20T13:10:57.217 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:10:57.226 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:10:57.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:10:57.227 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:10:57.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:10:57.228 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:10:57.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:10:57.232 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:12:01.050 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:12:01.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:12:01.058 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:12:01.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:12:01.052+0000 7fb3aaaee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58888) UID: 1000 2021-02-20T13:12:01.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:12:07.059 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:12:07.059 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:12:07.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:12:07.060 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:12:07.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:12:07.061 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:12:07.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:12:07.064 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T13:12:37.233 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:12:37.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:12:37.242 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:12:37.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:12:37.235+0000 7f0cb0fde700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59174) UID: 1000 2021-02-20T13:12:37.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:12:41.243 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:12:41.250 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:12:41.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:12:41.251 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:12:41.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:12:41.251 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:12:41.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:12:41.254 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:13:06.765 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:13:06.757+0000 7f508e15e700 -1 reset not still connected to 0x55ac85b9af20 2021-02-20T13:13:51.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6127104681993698) > weight (0.25) 2021-02-20T13:13:51.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.312429173159227) > weight (0.25) 2021-02-20T13:13:51.066 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:13:51.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:13:51.067 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:13:51.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:13:51.066+0000 7fcd33d08700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59083) UID: 1000 2021-02-20T13:13:51.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:13:56.069 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:13:56.070 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:13:56.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:13:56.071 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:13:56.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:13:56.071 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:13:56.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:13:56.075 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:14:06.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.637191493504858) > weight (0.25) 2021-02-20T13:14:06.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5532649591343514) > weight (0.25) 2021-02-20T13:14:06.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7490117196932037) > weight (0.25) 2021-02-20T13:14:06.256 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.975429334960333) > weight (0.25) 2021-02-20T13:14:06.256 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T13:14:56.076 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5475487870395457) > weight (0.25) 2021-02-20T13:14:56.086 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:14:56.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:14:56.087 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:14:56.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:14:56.084+0000 7fb091210700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58118) UID: 1000 2021-02-20T13:14:56.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:15:04.088 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:15:04.089 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:15:04.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:15:04.090 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:15:04.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:15:04.092 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:15:04.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:15:04.096 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T13:15:28.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T13:15:28.032+0000 7f5bfcd9c700 -1 rbd::mirror::ImageReplayer: 0x5615faf06c80 [3/eb5ee405-f45d-4109-b887-4acc3932b673] start: already running 2021-02-20T13:15:34.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3109016399005228) > weight (0.25) 2021-02-20T13:15:34.266 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:15:34.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:15:34.268 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:15:34.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:15:34.263+0000 7fbd02758700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 58402) UID: 1000 2021-02-20T13:15:34.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:15:39.269 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:15:39.270 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:15:39.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:15:39.271 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:15:39.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:15:39.272 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:15:39.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:15:39.276 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T13:16:28.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3490617132217114) > weight (0.25) 2021-02-20T13:16:28.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7484752515554227) > weight (0.25) 2021-02-20T13:16:28.107 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:16:28.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:16:28.109 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:16:28.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:16:28.106+0000 7ff4a51a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59566) UID: 1000 2021-02-20T13:16:28.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:16:34.110 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:16:34.118 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:16:34.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:16:34.119 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:16:34.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:16:34.120 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:16:34.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:16:34.123 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T13:17:31.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2664584680935972) > weight (0.25) 2021-02-20T13:17:31.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6641463973117638) > weight (0.25) 2021-02-20T13:17:31.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32486704745477657) > weight (0.25) 2021-02-20T13:17:31.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7137602477602) > weight (0.25) 2021-02-20T13:17:31.279 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T13:17:58.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5776955818347851) > weight (0.25) 2021-02-20T13:17:58.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5547027052804325) > weight (0.25) 2021-02-20T13:17:58.125 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:17:58.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:17:58.126 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:17:58.128 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:17:58.124+0000 7f8aed548700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59852) UID: 1000 2021-02-20T13:17:58.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:17:59.128 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:17:59.129 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:17:59.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:17:59.129 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:17:59.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:17:59.130 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:17:59.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:17:59.133 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T13:18:33.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9184301282959765) > weight (0.25) 2021-02-20T13:18:33.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5650639390009415) > weight (0.25) 2021-02-20T13:18:33.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8314302455057269) > weight (0.25) 2021-02-20T13:18:33.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2695716828688056) > weight (0.25) 2021-02-20T13:18:33.281 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T13:19:00.134 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9184995859153143) > weight (0.25) 2021-02-20T13:19:00.134 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:19:00.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:19:00.135 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:19:00.136 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:19:00.135+0000 7fe11506d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59661) UID: 1000 2021-02-20T13:19:00.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:19:02.137 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:19:02.138 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:19:02.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:19:02.139 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:19:02.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:19:02.140 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:19:02.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:19:02.144 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T13:20:19.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6194728704792688) > weight (0.25) 2021-02-20T13:20:19.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5002968991510546) > weight (0.25) 2021-02-20T13:20:19.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9306826561609982) > weight (0.25) 2021-02-20T13:20:19.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.500065656273368) > weight (0.25) 2021-02-20T13:20:19.291 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T13:20:44.145 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9105179281643004) > weight (0.25) 2021-02-20T13:20:44.153 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9210603261356874) > weight (0.25) 2021-02-20T13:20:44.153 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8264965615302867) > weight (0.25) 2021-02-20T13:20:44.153 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8055169923973858) > weight (0.25) 2021-02-20T13:20:44.154 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:21:44.154 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7489872789072044) > weight (0.25) 2021-02-20T13:21:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7752457366408699) > weight (0.25) 2021-02-20T13:21:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5710440986874932) > weight (0.25) 2021-02-20T13:21:44.155 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2702888350743865) > weight (0.25) 2021-02-20T13:21:44.155 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T13:21:45.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3617761666864807) > weight (0.25) 2021-02-20T13:21:45.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37341453060029295) > weight (0.25) 2021-02-20T13:21:45.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48895605835315337) > weight (0.25) 2021-02-20T13:21:45.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8628456844396948) > weight (0.25) 2021-02-20T13:21:45.294 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T13:23:04.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27709183467079046) > weight (0.25) 2021-02-20T13:23:04.163 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41385726356665264) > weight (0.25) 2021-02-20T13:23:04.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4518844931859224) > weight (0.25) 2021-02-20T13:23:04.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2518743133852951) > weight (0.25) 2021-02-20T13:23:04.164 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T13:23:14.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3185217036966763) > weight (0.25) 2021-02-20T13:23:14.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5628603472049754) > weight (0.25) 2021-02-20T13:23:14.303 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:23:14.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:23:14.305 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:23:14.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T13:23:14.302+0000 7f17b7a7e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 58212) UID: 1000 2021-02-20T13:23:14.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:23:16.038 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:23:16.034+0000 7f508e15e700 -1 reset not still connected to 0x55ac862caff0 2021-02-20T13:23:21.306 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:23:21.312 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:23:21.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:23:21.313 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:23:21.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:23:21.314 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:23:21.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:23:21.319 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:23:25.654 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:23:25.650+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f8a40 2021-02-20T13:24:51.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8947997712336917) > weight (0.25) 2021-02-20T13:24:51.196 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:24:51.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:24:51.198 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:24:51.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:24:51.195+0000 7f6624054700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60038) UID: 1000 2021-02-20T13:24:51.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:24:58.199 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:24:58.200 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:24:58.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:24:58.200 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:24:58.201 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:24:58.201 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:24:58.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:24:58.205 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T13:25:19.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35835456725775805) > weight (0.25) 2021-02-20T13:25:19.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7234597858798323) > weight (0.25) 2021-02-20T13:25:19.333 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:25:19.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:25:19.334 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:25:19.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:25:19.335 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:25:19.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T13:25:19.335+0000 7f3b60770700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60134) UID: 1000 2021-02-20T13:25:19.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:25:25.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:25:25.438 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:25:25.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:25:25.439 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:25:25.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:25:25.445 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T13:26:11.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.548800243372088) > weight (0.25) 2021-02-20T13:26:11.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5642190263743158) > weight (0.25) 2021-02-20T13:26:11.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.48729355109323114) > weight (0.25) 2021-02-20T13:26:11.220 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:26:11.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:26:11.221 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:26:11.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T13:26:11.218+0000 7f5bfd59d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 58502) UID: 1000 2021-02-20T13:26:11.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:26:15.222 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:26:15.231 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:26:15.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:26:15.232 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:26:15.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:26:15.233 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:26:15.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:26:15.239 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T13:26:39.489 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T13:26:39.485+0000 7f803c4bb700 -1 reset not still connected to 0x55accab03a80 2021-02-20T13:26:39.492 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T13:26:39.489+0000 7f803c4bb700 -1 reset not still connected to 0x55accab03c20 2021-02-20T13:26:45.447 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8727138749208483) > weight (0.25) 2021-02-20T13:26:45.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5954125505928577) > weight (0.25) 2021-02-20T13:26:45.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8764725900237792) > weight (0.25) 2021-02-20T13:26:45.457 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:26:45.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:26:45.458 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:26:45.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T13:26:45.457+0000 7f703e03a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 59276) UID: 1000 2021-02-20T13:26:45.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:26:51.460 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:26:51.461 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:26:51.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:26:51.463 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:26:51.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:26:51.464 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:26:51.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:26:51.469 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T13:27:17.021 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:27:17.017+0000 7f508e15e700 -1 reset not still connected to 0x55ac879747d0 2021-02-20T13:27:17.022 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:27:17.017+0000 7f508e15e700 -1 reset not still connected to 0x55ac87975740 2021-02-20T13:27:56.239 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:27:56.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:27:56.247 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:27:56.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:27:56.247+0000 7f9dcc9c4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59375) UID: 1000 2021-02-20T13:27:56.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:28:03.248 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:28:03.248 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:28:03.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:28:03.250 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:28:03.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:28:03.251 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:28:03.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:28:03.275 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:28:18.471 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9212253247442307) > weight (0.25) 2021-02-20T13:28:18.471 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3543549202992421) > weight (0.25) 2021-02-20T13:28:18.471 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:28:18.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:28:18.472 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:28:18.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T13:28:18.467+0000 7f20d804c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60329) UID: 1000 2021-02-20T13:28:18.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:28:22.473 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:28:22.473 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:28:22.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:28:22.474 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:28:22.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:28:22.475 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:28:22.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:28:22.478 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:29:43.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8270812449896573) > weight (0.25) 2021-02-20T13:29:43.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9116642807536833) > weight (0.25) 2021-02-20T13:29:43.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8137859031354211) > weight (0.25) 2021-02-20T13:29:43.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7083660722621182) > weight (0.25) 2021-02-20T13:29:43.280 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T13:30:17.480 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:30:17.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:30:17.482 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:30:17.482 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:30:17.482 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:30:17.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:30:17.480+0000 7f5c2ff05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59466) UID: 1000 2021-02-20T13:30:17.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:30:23.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:30:23.586 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:30:23.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:30:23.587 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:30:23.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:30:23.590 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T13:31:03.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5933785139027591) > weight (0.25) 2021-02-20T13:31:03.296 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:31:03.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:31:03.299 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:31:03.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:31:03.295+0000 7f84d9953700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60234) UID: 1000 2021-02-20T13:31:03.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:31:10.300 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:31:10.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:31:10.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:31:10.308 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:31:10.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:31:10.309 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:31:10.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:31:10.312 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:31:27.591 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.90582113052659) > weight (0.25) 2021-02-20T13:31:27.599 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6379424059671667) > weight (0.25) 2021-02-20T13:31:27.599 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37053770288472987) > weight (0.25) 2021-02-20T13:31:27.600 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3449366507862266) > weight (0.25) 2021-02-20T13:31:27.600 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T13:33:05.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7148598869295305) > weight (0.25) 2021-02-20T13:33:05.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.45843259400378467) > weight (0.25) 2021-02-20T13:33:05.327 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:33:05.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:33:05.329 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:33:05.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:33:05.329+0000 7f19b7ff0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59947) UID: 1000 2021-02-20T13:33:05.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:33:06.602 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:33:06.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:33:06.603 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:33:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:33:06.601+0000 7f6a7a675700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60812) UID: 1000 2021-02-20T13:33:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:33:08.330 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:33:08.331 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:33:08.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:33:08.332 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:33:08.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:33:08.332 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:33:08.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:33:08.335 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T13:33:13.604 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:33:13.605 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:33:13.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:33:13.606 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:33:13.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:33:13.606 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:33:13.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:33:13.610 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T13:34:20.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.897945933359142) > weight (0.25) 2021-02-20T13:34:20.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5694249742358563) > weight (0.25) 2021-02-20T13:34:20.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.36839248640954914) > weight (0.25) 2021-02-20T13:34:20.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9305180648960534) > weight (0.25) 2021-02-20T13:34:20.624 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T13:34:22.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8375108261432078) > weight (0.25) 2021-02-20T13:34:22.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5514750227660785) > weight (0.25) 2021-02-20T13:34:22.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8495955536270077) > weight (0.25) 2021-02-20T13:34:22.338 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T13:34:22.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T13:34:22.339 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:34:22.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T13:34:22.335+0000 7ff9bb72b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 60429) UID: 1000 2021-02-20T13:34:22.741 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:34:25.340 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T13:34:25.340 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:34:25.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:34:25.341 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T13:34:25.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T13:34:25.342 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T13:34:25.345 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T13:34:25.345 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:35:55.626 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:35:55.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:35:55.636 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T13:35:55.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:35:55.633+0000 7ff442297700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61095) UID: 1000 2021-02-20T13:35:56.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:36:01.637 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:36:01.649 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:36:01.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:36:01.650 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:36:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:36:01.651 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:36:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:36:01.655 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T13:36:20.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9484992237641836) > weight (0.25) 2021-02-20T13:36:20.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.409096247250323) > weight (0.25) 2021-02-20T13:36:20.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5057005944654722) > weight (0.25) 2021-02-20T13:36:20.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.42018173941841075) > weight (0.25) 2021-02-20T13:36:20.360 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T13:36:33.542 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:36:33.536+0000 7f508e15e700 -1 reset not still connected to 0x55ac862ca490 2021-02-20T13:37:17.656 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:37:17.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:37:17.657 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:37:17.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:37:17.655+0000 7f1395ac7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61289) UID: 1000 2021-02-20T13:37:18.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:37:24.658 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:37:24.659 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:37:24.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:37:24.660 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:37:24.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:37:24.660 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:37:24.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:37:24.663 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T13:37:28.537 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:37:28.534+0000 7f508e15e700 -1 reset not still connected to 0x55ac87975cf0 2021-02-20T13:37:48.486 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:37:48.482+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3a8f0 2021-02-20T13:37:48.487 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:37:48.482+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3d8e0 2021-02-20T13:37:48.487 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:37:48.482+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3da80 2021-02-20T13:37:48.487 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:37:48.482+0000 7f508e15e700 -1 reset not still connected to 0x55ac87973e10 2021-02-20T13:38:07.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3620451219959453) > weight (0.25) 2021-02-20T13:38:07.361 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:38:07.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:38:07.362 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:38:07.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:38:07.362+0000 7fce20d4a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60911) UID: 1000 2021-02-20T13:38:07.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:38:14.363 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:38:14.364 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:38:14.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:38:14.365 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:38:14.365 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:38:14.365 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:38:14.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:38:14.368 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T13:39:05.664 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:39:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:39:05.666 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:39:05.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:39:05.665+0000 7f4ee8e99700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61390) UID: 1000 2021-02-20T13:39:06.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:39:12.666 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:39:12.667 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:39:12.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:39:12.668 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:39:12.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:39:12.669 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:39:12.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:39:12.672 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T13:39:27.374 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33912135314722436) > weight (0.25) 2021-02-20T13:39:27.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.82079245291491) > weight (0.25) 2021-02-20T13:39:27.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37514422384882284) > weight (0.25) 2021-02-20T13:39:27.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4200552348973986) > weight (0.25) 2021-02-20T13:39:27.376 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T13:39:38.578 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:39:38.571+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e23740 2021-02-20T13:40:33.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29771326715633806) > weight (0.25) 2021-02-20T13:40:33.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4978353175367901) > weight (0.25) 2021-02-20T13:40:33.394 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:40:33.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:40:33.395 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:40:33.396 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:40:33.395+0000 7fa636813700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61004) UID: 1000 2021-02-20T13:40:33.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:40:38.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:40:38.397 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:40:38.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:40:38.398 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:40:38.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:40:38.398 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:40:38.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:40:38.401 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T13:40:41.741 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:40:41.738+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e22150 2021-02-20T13:41:00.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7322826785672644) > weight (0.25) 2021-02-20T13:41:00.673 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:41:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:41:00.675 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T13:41:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:41:00.674+0000 7f4df9a3b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59753) UID: 1000 2021-02-20T13:41:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:41:02.441 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:41:02.433+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3a8f0 2021-02-20T13:41:02.441 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:41:02.433+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3da80 2021-02-20T13:41:02.443 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:41:02.437+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750c150 2021-02-20T13:41:02.444 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:41:02.437+0000 7f508e15e700 -1 reset not still connected to 0x55ac87973e10 2021-02-20T13:41:02.444 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:41:02.437+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e21fb0 2021-02-20T13:41:05.676 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:41:05.682 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:41:05.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:41:05.683 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:41:05.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:41:05.684 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:41:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:41:05.687 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:42:24.403 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:42:24.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:42:24.405 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:42:24.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:42:24.405+0000 7fd42abee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60620) UID: 1000 2021-02-20T13:42:24.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:42:26.406 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:42:26.406 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:42:26.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:42:26.407 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:42:26.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:42:26.408 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:42:26.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:42:26.411 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T13:42:45.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30727791605138266) > weight (0.25) 2021-02-20T13:42:45.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7578482111108322) > weight (0.25) 2021-02-20T13:42:45.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9332126483932985) > weight (0.25) 2021-02-20T13:42:45.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8115656794978025) > weight (0.25) 2021-02-20T13:42:45.691 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T13:43:26.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6890306639984949) > weight (0.25) 2021-02-20T13:43:26.413 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:43:26.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:43:26.415 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:43:26.415 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:43:26.415+0000 7f59a41a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61489) UID: 1000 2021-02-20T13:43:26.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:43:34.415 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:43:34.416 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:43:34.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:43:34.417 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:43:34.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:43:34.418 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:43:34.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:43:34.422 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T13:44:26.691 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:44:26.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:44:26.698 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T13:44:26.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:44:26.698+0000 7f3d980de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61580) UID: 1000 2021-02-20T13:44:27.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:44:34.699 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:44:34.700 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:44:34.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:44:34.700 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:44:34.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:44:34.701 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:44:34.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:44:34.704 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T13:44:45.422 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:44:45.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:44:45.433 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T13:44:45.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:44:45.434+0000 7fd20152e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61872) UID: 1000 2021-02-20T13:44:45.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:44:46.434 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:44:46.434 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:44:46.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:44:46.437 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:44:46.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:44:46.440 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:44:46.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:44:46.446 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T13:46:08.447 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8374959214955507) > weight (0.25) 2021-02-20T13:46:08.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7531366593879683) > weight (0.25) 2021-02-20T13:46:08.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8107400233982298) > weight (0.25) 2021-02-20T13:46:08.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3107992258314489) > weight (0.25) 2021-02-20T13:46:08.449 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T13:46:29.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5857382291516371) > weight (0.25) 2021-02-20T13:46:29.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8309260369688626) > weight (0.25) 2021-02-20T13:46:29.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5183900331268461) > weight (0.25) 2021-02-20T13:46:29.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7818275870495011) > weight (0.25) 2021-02-20T13:46:29.706 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T13:47:42.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8498259610744116) > weight (0.25) 2021-02-20T13:47:42.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4411392405020176) > weight (0.25) 2021-02-20T13:47:42.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5430055730652786) > weight (0.25) 2021-02-20T13:47:42.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8012216187363626) > weight (0.25) 2021-02-20T13:47:42.452 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T13:48:10.707 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T13:48:10.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T13:48:10.710 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T13:48:10.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T13:48:10.709+0000 7f2b41218700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62057) UID: 1000 2021-02-20T13:48:10.720 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T13:48:10.719+0000 7f62c7ffc700 -1 reset not still connected to 0x555b560a8560 2021-02-20T13:48:11.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:48:14.710 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T13:48:14.710 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:48:14.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:48:14.712 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T13:48:14.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T13:48:14.712 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T13:48:14.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T13:48:14.716 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T13:49:07.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9733470830428431) > weight (0.25) 2021-02-20T13:49:07.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5536100079453415) > weight (0.25) 2021-02-20T13:49:07.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4278558082736067) > weight (0.25) 2021-02-20T13:49:07.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.49337788129602755) > weight (0.25) 2021-02-20T13:49:07.454 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T13:49:38.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5944812478715708) > weight (0.25) 2021-02-20T13:49:38.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7092906858749024) > weight (0.25) 2021-02-20T13:49:38.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.38452701042709525) > weight (0.25) 2021-02-20T13:49:38.718 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3855154374163481) > weight (0.25) 2021-02-20T13:49:38.718 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T13:50:45.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33760373588157544) > weight (0.25) 2021-02-20T13:50:45.720 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:50:45.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:50:45.721 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:50:45.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:50:45.722+0000 7fbca57df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61772) UID: 1000 2021-02-20T13:50:46.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:50:48.723 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:50:48.723 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:50:48.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:50:48.724 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:50:48.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:50:48.725 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:50:48.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:50:48.728 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T13:51:00.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9596015096298901) > weight (0.25) 2021-02-20T13:51:00.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9864859786066522) > weight (0.25) 2021-02-20T13:51:00.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6054273436575413) > weight (0.25) 2021-02-20T13:51:00.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.887711077624512) > weight (0.25) 2021-02-20T13:51:00.456 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T13:51:13.680 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:51:13.679+0000 7f508e15e700 -1 reset not still connected to 0x55ac87975740 2021-02-20T13:51:13.681 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:51:13.679+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f6f70 2021-02-20T13:52:14.729 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47385723610204133) > weight (0.25) 2021-02-20T13:52:14.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8853296205005834) > weight (0.25) 2021-02-20T13:52:14.730 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T13:52:14.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T13:52:14.731 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:52:14.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T13:52:14.732+0000 7f9a30b84700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60711) UID: 1000 2021-02-20T13:52:15.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:52:21.732 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T13:52:21.743 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:52:21.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:52:21.744 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T13:52:21.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T13:52:21.745 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T13:52:21.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T13:52:21.749 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T13:52:57.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9217193742819851) > weight (0.25) 2021-02-20T13:52:57.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5100246340898626) > weight (0.25) 2021-02-20T13:52:57.459 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T13:52:57.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T13:52:57.460 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:52:57.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T13:52:57.459+0000 7f42d60af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61680) UID: 1000 2021-02-20T13:52:57.595 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T13:52:57.591+0000 7f803c4bb700 -1 reset not still connected to 0x55accf695e10 2021-02-20T13:52:57.595 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T13:52:57.591+0000 7f803c4bb700 -1 reset not still connected to 0x55accf697e90 2021-02-20T13:52:57.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:53:00.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T13:53:00.461 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:53:00.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:53:00.462 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T13:53:00.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T13:53:00.463 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T13:53:00.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T13:53:00.466 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T13:54:06.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6670014267072779) > weight (0.25) 2021-02-20T13:54:06.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9120070186430823) > weight (0.25) 2021-02-20T13:54:06.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3877653879599521) > weight (0.25) 2021-02-20T13:54:06.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.98578554158454) > weight (0.25) 2021-02-20T13:54:06.758 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T13:54:51.467 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T13:54:51.468 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T13:54:51.469 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:54:51.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T13:54:51.469+0000 7f05a0823700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62156) UID: 1000 2021-02-20T13:54:51.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:54:54.470 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T13:54:54.470 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:54:54.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:54:54.472 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T13:54:54.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T13:54:54.473 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T13:54:54.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T13:54:54.477 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T13:56:04.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6624337870662151) > weight (0.25) 2021-02-20T13:56:04.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.35162039234507037) > weight (0.25) 2021-02-20T13:56:04.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6042471797975473) > weight (0.25) 2021-02-20T13:56:04.763 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T13:56:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T13:56:04.765 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T13:56:04.765 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T13:56:04.765 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:56:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T13:56:04.759+0000 7f0d712be700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 60521) UID: 1000 2021-02-20T13:56:05.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:56:10.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:56:10.874 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T13:56:10.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T13:56:10.875 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T13:56:10.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T13:56:10.878 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T13:56:34.478 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3245375384471776) > weight (0.25) 2021-02-20T13:56:34.485 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:56:34.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:56:34.486 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T13:56:34.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:56:34.486+0000 7f4d4e885700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 61965) UID: 1000 2021-02-20T13:56:34.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:56:36.486 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:56:36.487 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:56:36.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:56:36.487 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:56:36.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:56:36.488 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:56:36.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:56:36.491 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T13:57:43.879 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3083151464217271) > weight (0.25) 2021-02-20T13:57:43.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:57:43.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:57:43.887 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T13:57:43.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:57:43.885+0000 7f9c92524700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62354) UID: 1000 2021-02-20T13:57:44.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:57:45.291 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:57:45.290+0000 7f508e15e700 -1 reset not still connected to 0x55ac8295be10 2021-02-20T13:57:45.292 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:57:45.290+0000 7f508e15e700 -1 reset not still connected to 0x55ac8295ca40 2021-02-20T13:57:45.292 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T13:57:45.290+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3d260 2021-02-20T13:57:46.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:57:46.888 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:57:46.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:57:46.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:57:46.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:57:46.890 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:57:46.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:57:46.893 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T13:58:15.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35089499720016104) > weight (0.25) 2021-02-20T13:58:15.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8913583608925904) > weight (0.25) 2021-02-20T13:58:15.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5254563061258576) > weight (0.25) 2021-02-20T13:58:15.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7479667096011279) > weight (0.25) 2021-02-20T13:58:15.493 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T13:59:23.894 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7444944891779316) > weight (0.25) 2021-02-20T13:59:23.895 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T13:59:23.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T13:59:23.896 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T13:59:23.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T13:59:23.895+0000 7f6016e1d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62952) UID: 1000 2021-02-20T13:59:24.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:59:32.898 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T13:59:32.898 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:59:32.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:59:32.900 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T13:59:32.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T13:59:32.901 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T13:59:32.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T13:59:32.905 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T13:59:47.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26902114965242907) > weight (0.25) 2021-02-20T13:59:47.494 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T13:59:47.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T13:59:47.495 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T13:59:47.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T13:59:47.494+0000 7f4a50895700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62858) UID: 1000 2021-02-20T13:59:47.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T13:59:54.496 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T13:59:54.496 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T13:59:54.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T13:59:54.497 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T13:59:54.497 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T13:59:54.498 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T13:59:54.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T13:59:54.501 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:00:56.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4081833606383891) > weight (0.25) 2021-02-20T14:00:56.512 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5041789220507236) > weight (0.25) 2021-02-20T14:00:56.512 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:00:56.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:00:56.514 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:00:56.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T14:00:56.513+0000 7f306b40e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 62573) UID: 1000 2021-02-20T14:00:56.916 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:00:57.515 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:00:57.516 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:00:57.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:00:57.517 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:00:57.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:00:57.519 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:00:57.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:00:57.523 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T14:01:16.906 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:01:16.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:01:16.920 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:01:16.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:01:16.916+0000 7f9d9f048700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62252) UID: 1000 2021-02-20T14:01:17.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:01:25.921 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:01:25.931 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:01:25.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:01:25.932 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:01:25.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:01:25.933 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:01:25.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:01:25.937 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:02:33.093 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:02:33.087+0000 7f508e15e700 -1 reset not still connected to 0x55ac7f0f2220 2021-02-20T14:02:33.094 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:02:33.087+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c9040 2021-02-20T14:02:33.095 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:02:33.087+0000 7f508e15e700 -1 reset not still connected to 0x55ac862cb8e0 2021-02-20T14:02:33.095 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:02:33.091+0000 7f508e15e700 -1 reset not still connected to 0x55ac87272000 2021-02-20T14:02:34.524 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:02:34.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:02:34.535 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:02:34.535 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:02:34.535 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:02:34.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:02:34.535+0000 7fce14b6e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62665) UID: 1000 2021-02-20T14:02:34.937 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:02:40.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:02:40.639 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:02:40.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:02:40.639 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:02:40.642 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:02:40.642 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:03:18.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6360344803694047) > weight (0.25) 2021-02-20T14:03:18.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.364030453551219) > weight (0.25) 2021-02-20T14:03:18.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6126094623193072) > weight (0.25) 2021-02-20T14:03:18.949 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.536499560153679) > weight (0.25) 2021-02-20T14:03:18.950 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T14:03:51.643 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9278348809223173) > weight (0.25) 2021-02-20T14:03:51.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27780271721105465) > weight (0.25) 2021-02-20T14:03:51.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8319917521238082) > weight (0.25) 2021-02-20T14:03:51.645 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:03:51.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:03:51.646 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:03:51.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:03:51.645+0000 7f37433a6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 61194) UID: 1000 2021-02-20T14:03:52.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:03:57.647 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:03:57.648 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:03:57.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:03:57.649 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:03:57.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:03:57.650 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:03:57.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:03:57.654 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T14:04:53.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5137000843161692) > weight (0.25) 2021-02-20T14:04:53.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6462055570760629) > weight (0.25) 2021-02-20T14:04:53.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5426902140895007) > weight (0.25) 2021-02-20T14:04:53.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6560578627605804) > weight (0.25) 2021-02-20T14:04:53.962 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T14:05:11.655 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:05:11.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:05:11.669 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:05:11.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:05:11.675+0000 7f53abd2c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63433) UID: 1000 2021-02-20T14:05:12.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:05:15.670 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:05:15.676 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:05:15.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:05:15.677 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:05:15.678 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:05:15.678 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:05:15.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:05:15.681 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T14:06:16.682 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:06:16.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:06:16.693 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:06:16.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:06:16.694+0000 7ff1d2e03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63622) UID: 1000 2021-02-20T14:06:17.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:06:24.695 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:06:24.695 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:06:24.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:06:24.696 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:06:24.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:06:24.697 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:06:24.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:06:24.700 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:06:26.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5947279282331904) > weight (0.25) 2021-02-20T14:06:26.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8170078932010537) > weight (0.25) 2021-02-20T14:06:26.963 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9505900219205139) > weight (0.25) 2021-02-20T14:06:26.963 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:06:26.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:06:26.964 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:06:26.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T14:06:26.962+0000 7f191a1b7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 62759) UID: 1000 2021-02-20T14:06:27.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:06:28.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x56009f256f00 [3/674f23ca-7630-45f2-9a82-095e98b79c76] start: already running 2021-02-20T14:06:28.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x56009f257180 [3/75792560-55ec-454a-bb3c-f7a23e6221d8] start: already running 2021-02-20T14:06:28.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x56009f257400 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T14:06:28.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x5600a3447180 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T14:06:28.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x56009f257b80 [3/9a469893-68d6-4d91-a371-3899fcc47454] start: already running 2021-02-20T14:06:28.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.218+0000 7fe80ab8e700 -1 rbd::mirror::ImageReplayer: 0x56009f257e00 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T14:06:28.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:06:28.230+0000 7fe80b38f700 -1 rbd::mirror::ImageReplayer: 0x56009f257e00 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T14:06:30.965 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:06:30.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:06:30.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:06:30.966 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:06:30.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:06:30.967 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:06:30.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:06:30.970 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T14:07:48.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9403110091707096) > weight (0.25) 2021-02-20T14:07:48.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4975906343907617) > weight (0.25) 2021-02-20T14:07:48.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30507175303273015) > weight (0.25) 2021-02-20T14:07:48.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8639765655940442) > weight (0.25) 2021-02-20T14:07:48.972 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T14:07:51.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7984600225975913) > weight (0.25) 2021-02-20T14:07:51.701 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:07:51.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:07:51.703 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:07:51.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:07:51.700+0000 7f5ade660700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 63152) UID: 1000 2021-02-20T14:07:52.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:07:58.704 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:07:58.704 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:07:58.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:07:58.705 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:07:58.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:07:58.706 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:07:58.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:07:58.715 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:09:46.974 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:09:47.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:09:47.005 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:09:47.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:09:47.001+0000 7f8ba897e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63334) UID: 1000 2021-02-20T14:09:47.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:09:51.005 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:09:51.006 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:09:51.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:09:51.007 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:09:51.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:09:51.007 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:09:51.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:09:51.012 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:09:51.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5116748504781125) > weight (0.25) 2021-02-20T14:09:51.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7656858700158496) > weight (0.25) 2021-02-20T14:09:51.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27592159621570167) > weight (0.25) 2021-02-20T14:09:51.717 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:09:51.718 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:09:51.718 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:09:51.718 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:09:51.718 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:09:51.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:09:51.717+0000 7f3c44017700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 63526) UID: 1000 2021-02-20T14:09:52.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:09:57.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:09:57.821 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:09:57.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:09:57.822 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:09:57.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:09:57.824 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:11:35.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7511487017048799) > weight (0.25) 2021-02-20T14:11:35.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4627924836346593) > weight (0.25) 2021-02-20T14:11:35.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8980096361940315) > weight (0.25) 2021-02-20T14:11:35.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8928369801450589) > weight (0.25) 2021-02-20T14:11:35.838 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T14:11:43.012 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9810900371192457) > weight (0.25) 2021-02-20T14:11:43.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3042011013800431) > weight (0.25) 2021-02-20T14:11:43.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9183310439464327) > weight (0.25) 2021-02-20T14:11:43.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8228766202676604) > weight (0.25) 2021-02-20T14:11:43.019 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T14:12:45.021 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:12:45.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:12:45.030 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:12:45.031 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:12:45.031 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:12:45.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:12:45.029+0000 7f82841aa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 63996) UID: 1000 2021-02-20T14:12:45.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:12:51.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:12:51.162 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:12:51.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:12:51.168 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:12:51.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:12:51.192 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T14:12:56.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7609159483696295) > weight (0.25) 2021-02-20T14:12:56.844 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8356975607624316) > weight (0.25) 2021-02-20T14:12:56.845 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:12:56.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:12:56.846 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:12:56.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T14:12:56.845+0000 7feb71d49700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 63243) UID: 1000 2021-02-20T14:12:57.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:13:03.846 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:13:03.847 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:13:03.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:13:03.848 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:13:03.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:13:03.848 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:13:03.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:13:03.851 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T14:13:15.687 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:13:15.685+0000 7f803c4bb700 -1 reset not still connected to 0x55acdacf6490 2021-02-20T14:14:32.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.773187184056841) > weight (0.25) 2021-02-20T14:14:32.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9248882339528515) > weight (0.25) 2021-02-20T14:14:32.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3314655414797265) > weight (0.25) 2021-02-20T14:14:32.860 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:14:32.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:14:32.862 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:14:32.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:14:32.859+0000 7eff51f55700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64096) UID: 1000 2021-02-20T14:14:33.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:14:34.862 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:14:34.863 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:14:34.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:14:34.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:14:34.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:14:34.864 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:14:34.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:14:34.867 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T14:14:37.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7388166327729481) > weight (0.25) 2021-02-20T14:14:37.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3142718138556704) > weight (0.25) 2021-02-20T14:14:37.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8715205827932817) > weight (0.25) 2021-02-20T14:14:37.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9723855236123988) > weight (0.25) 2021-02-20T14:14:37.195 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T14:15:41.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9854135787935596) > weight (0.25) 2021-02-20T14:15:41.869 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:15:41.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:15:41.871 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:15:41.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:15:41.870+0000 7faf30537700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 63903) UID: 1000 2021-02-20T14:15:42.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:15:42.872 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:15:42.872 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:15:42.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:15:42.873 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:15:42.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:15:42.874 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:15:42.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:15:42.876 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T14:15:44.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8691111299665264) > weight (0.25) 2021-02-20T14:15:44.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8399870391095625) > weight (0.25) 2021-02-20T14:15:44.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.809293182242726) > weight (0.25) 2021-02-20T14:15:44.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.647064298774497) > weight (0.25) 2021-02-20T14:15:44.197 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T14:17:09.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.456580473633837) > weight (0.25) 2021-02-20T14:17:09.878 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:17:09.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:17:09.880 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:17:09.880 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:17:09.880 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:17:09.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:17:09.876+0000 7fc73c253700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 64476) UID: 1000 2021-02-20T14:17:10.198 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:17:10.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:17:10.199 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:17:10.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:17:10.196+0000 7fc058538700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64189) UID: 1000 2021-02-20T14:17:10.281 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:17:10.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:17:15.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:17:15.986 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:17:15.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:17:15.989 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:17:15.996 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:17:15.997 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:17:19.200 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:17:19.200 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:17:19.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:17:19.201 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:17:19.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:17:19.202 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:17:19.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:17:19.204 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T14:18:26.998 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6191239212531867) > weight (0.25) 2021-02-20T14:18:27.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8550923060407001) > weight (0.25) 2021-02-20T14:18:27.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2862354858603978) > weight (0.25) 2021-02-20T14:18:27.362 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:18:27.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:18:27.369 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:18:27.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:18:27.370+0000 7fa8dc035700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64380) UID: 1000 2021-02-20T14:18:27.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:18:28.370 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:18:28.370 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:18:28.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:18:28.371 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:18:28.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:18:28.372 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:18:28.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:18:28.375 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T14:18:57.207 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4818091114420304) > weight (0.25) 2021-02-20T14:18:57.214 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:18:57.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:18:57.215 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:18:57.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:18:57.213+0000 7fe80bb90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 63053) UID: 1000 2021-02-20T14:18:57.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:19:04.216 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:19:04.216 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:19:04.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:19:04.217 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:19:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:19:04.218 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:19:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:19:04.221 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T14:20:26.375 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7450396956610453) > weight (0.25) 2021-02-20T14:20:26.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25907945168312074) > weight (0.25) 2021-02-20T14:20:26.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5818527503700752) > weight (0.25) 2021-02-20T14:20:26.411 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38088272205876317) > weight (0.25) 2021-02-20T14:20:26.411 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T14:21:00.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7249929500108218) > weight (0.25) 2021-02-20T14:21:00.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40497010096862907) > weight (0.25) 2021-02-20T14:21:00.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9827910866420612) > weight (0.25) 2021-02-20T14:21:00.242 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:21:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:21:00.244 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:21:00.244 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:21:00.244 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:21:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T14:21:00.243+0000 7f0ce37bd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 63804) UID: 1000 2021-02-20T14:21:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:21:06.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:21:06.347 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:21:06.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:21:06.348 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:21:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:21:06.355 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:21:32.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9881084679838816) > weight (0.25) 2021-02-20T14:21:32.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9747187841611842) > weight (0.25) 2021-02-20T14:21:32.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.406780577703157) > weight (0.25) 2021-02-20T14:21:32.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8588272516027279) > weight (0.25) 2021-02-20T14:21:32.413 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T14:22:56.357 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:22:56.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:22:56.358 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T14:22:56.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:22:56.356+0000 7f0d17b67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64661) UID: 1000 2021-02-20T14:22:56.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:23:04.359 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:23:04.359 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:23:04.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:23:04.360 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:23:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:23:04.361 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:23:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:23:04.365 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T14:23:09.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5592165765173874) > weight (0.25) 2021-02-20T14:23:09.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7025432026455658) > weight (0.25) 2021-02-20T14:23:09.415 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6410921464759941) > weight (0.25) 2021-02-20T14:23:09.415 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:23:09.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:23:09.416 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T14:23:09.417 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:23:09.412+0000 7faafec97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64760) UID: 1000 2021-02-20T14:23:09.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:23:11.417 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:23:11.417 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:23:11.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:23:11.419 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:23:11.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:23:11.419 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:23:11.423 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:23:11.423 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T14:24:17.366 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:24:17.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:24:17.376 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:24:17.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:24:17.375+0000 7f081edb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65054) UID: 1000 2021-02-20T14:24:17.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:24:26.376 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:24:26.384 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:24:26.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:24:26.385 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:24:26.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:24:26.386 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:24:26.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:24:26.389 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T14:24:34.426 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:24:34.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:24:34.427 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:24:34.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:24:34.426+0000 7f4a77aab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63713) UID: 1000 2021-02-20T14:24:34.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:24:35.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.698+0000 7f77de651700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6d680 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T14:24:35.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.702+0000 7f77dee52700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6d400 [3/6a6f5866-70c0-4063-8b45-7c49d8fcdf4b] start: already running 2021-02-20T14:24:35.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.702+0000 7f77de651700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6d900 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T14:24:35.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.702+0000 7f77de651700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6cc80 [3/7cc83948-bf26-4ed7-a142-2db0f2786e6e] start: already running 2021-02-20T14:24:35.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.706+0000 7f77dee52700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6d180 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T14:24:35.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:24:35.706+0000 7f77dee52700 -1 rbd::mirror::ImageReplayer: 0x55ea60e6db80 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T14:24:40.432 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:24:40.433 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:24:40.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:24:40.433 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:24:40.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:24:40.434 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:24:40.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:24:40.437 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T14:25:53.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9835333925282009) > weight (0.25) 2021-02-20T14:25:53.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4893868125916051) > weight (0.25) 2021-02-20T14:25:53.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46013643092422996) > weight (0.25) 2021-02-20T14:25:53.446 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7946661687793755) > weight (0.25) 2021-02-20T14:25:53.446 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:26:07.393 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:26:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:26:07.402 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:26:07.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:26:07.400+0000 7fcf14f9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65244) UID: 1000 2021-02-20T14:26:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:26:10.404 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:26:10.405 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:26:10.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:26:10.407 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:26:10.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:26:10.410 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:26:10.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:26:10.432 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T14:27:36.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9809276157252653) > weight (0.25) 2021-02-20T14:27:36.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5572438830608478) > weight (0.25) 2021-02-20T14:27:36.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8771682961390836) > weight (0.25) 2021-02-20T14:27:36.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6957498345399479) > weight (0.25) 2021-02-20T14:27:36.456 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T14:27:57.433 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4105707573006656) > weight (0.25) 2021-02-20T14:27:57.462 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:27:57.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:27:57.464 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:27:57.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:27:57.462+0000 7ff8dd308700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 64852) UID: 1000 2021-02-20T14:27:57.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:28:04.466 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:28:05.410 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:28:05.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:28:05.411 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:28:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:28:05.412 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:28:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:28:05.417 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T14:28:43.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30839864340684786) > weight (0.25) 2021-02-20T14:28:43.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.47358775505021566) > weight (0.25) 2021-02-20T14:28:43.466 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:28:43.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:28:43.467 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:28:43.468 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T14:28:43.465+0000 7f2ed85a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 64288) UID: 1000 2021-02-20T14:28:43.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:28:50.469 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:28:50.477 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:28:50.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:28:50.478 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:28:50.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:28:50.479 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:28:50.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:28:50.482 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T14:29:37.419 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6697630953676218) > weight (0.25) 2021-02-20T14:29:37.426 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7300058964305697) > weight (0.25) 2021-02-20T14:29:37.426 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:29:37.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:29:37.427 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:29:37.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T14:29:37.428+0000 7f1c12046700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 62454) UID: 1000 2021-02-20T14:29:37.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:29:44.428 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:29:44.429 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:29:44.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:29:44.430 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:29:44.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:29:44.430 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:29:44.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:29:44.434 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T14:30:32.482 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:30:34.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:30:34.153 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:30:34.154 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:30:34.150+0000 7fad1ab75700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65343) UID: 1000 2021-02-20T14:30:34.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:30:39.154 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:30:39.154 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:30:39.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:30:39.155 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:30:39.155 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:30:39.156 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:30:39.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:30:39.162 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T14:30:58.435 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:30:58.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:30:58.436 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:30:58.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:30:58.434+0000 7f5dd123a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65439) UID: 1000 2021-02-20T14:30:58.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:31:05.436 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:31:05.437 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:31:05.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:31:05.438 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:31:05.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:31:05.438 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:31:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:31:05.441 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T14:32:23.164 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:32:23.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:32:23.172 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:32:23.173 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:32:23.173 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:32:23.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:32:23.172+0000 7f6bf6c08700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65832) UID: 1000 2021-02-20T14:32:23.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:32:24.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9429209123901411) > weight (0.25) 2021-02-20T14:32:24.443 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:32:24.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:32:24.444 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:32:24.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:32:24.440+0000 7f00e8892700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65539) UID: 1000 2021-02-20T14:32:24.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:32:29.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:32:29.282 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:32:29.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:32:29.282 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:32:29.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:32:29.286 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T14:32:29.444 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:32:29.445 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:32:29.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:32:29.446 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:32:29.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:32:29.447 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:32:29.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:32:29.450 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T14:33:30.287 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:33:30.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:33:30.299 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:33:30.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:33:30.299+0000 7f7d085e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66024) UID: 1000 2021-02-20T14:33:30.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:33:31.300 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:33:31.301 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:33:31.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:33:31.303 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:33:31.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:33:31.304 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:33:31.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:33:31.322 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T14:33:38.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5891368867733853) > weight (0.25) 2021-02-20T14:33:38.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45943996101979157) > weight (0.25) 2021-02-20T14:33:38.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.45815720601750376) > weight (0.25) 2021-02-20T14:33:38.452 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:33:38.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:33:38.453 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:33:38.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T14:33:38.450+0000 7fe1707db700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 64953) UID: 1000 2021-02-20T14:33:38.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:33:43.454 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:33:43.454 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:33:43.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:33:43.455 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:33:43.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:33:43.456 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:33:43.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:33:43.465 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T14:34:54.323 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:34:54.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:34:54.330 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T14:34:54.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:34:54.329+0000 7f0a9b0b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66214) UID: 1000 2021-02-20T14:34:54.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:34:57.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5446446312480976) > weight (0.25) 2021-02-20T14:34:57.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2797080792168585) > weight (0.25) 2021-02-20T14:34:57.466 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5765940399353479) > weight (0.25) 2021-02-20T14:34:57.466 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:34:57.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:34:57.467 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:34:57.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T14:34:57.465+0000 7fdec9875700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66305) UID: 1000 2021-02-20T14:34:57.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:35:01.468 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:35:01.478 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:35:01.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:35:01.479 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:35:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:35:01.480 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:35:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:35:01.483 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T14:35:03.331 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:35:03.336 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:35:03.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:35:03.337 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:35:03.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:35:03.337 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:35:03.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:35:03.345 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T14:36:06.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6406736076312582) > weight (0.25) 2021-02-20T14:36:06.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.977514607817747) > weight (0.25) 2021-02-20T14:36:06.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8191317509774165) > weight (0.25) 2021-02-20T14:36:06.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8717942257992931) > weight (0.25) 2021-02-20T14:36:06.357 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T14:36:47.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49568151843502284) > weight (0.25) 2021-02-20T14:36:47.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5989831912173371) > weight (0.25) 2021-02-20T14:36:47.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6336578688677386) > weight (0.25) 2021-02-20T14:36:47.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4977178924670339) > weight (0.25) 2021-02-20T14:36:47.493 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T14:37:34.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7310096689319908) > weight (0.25) 2021-02-20T14:37:34.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.49136464404384717) > weight (0.25) 2021-02-20T14:37:34.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9407313210762913) > weight (0.25) 2021-02-20T14:37:34.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3773934559921772) > weight (0.25) 2021-02-20T14:37:34.359 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T14:38:43.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5966965991035825) > weight (0.25) 2021-02-20T14:38:43.504 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49715857981612555) > weight (0.25) 2021-02-20T14:38:43.504 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9447369338444486) > weight (0.25) 2021-02-20T14:38:43.504 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.296239459896554) > weight (0.25) 2021-02-20T14:38:43.505 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T14:39:11.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7000484489256379) > weight (0.25) 2021-02-20T14:39:11.383 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:39:11.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:39:11.387 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:39:11.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:39:11.383+0000 7f77df653700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 64570) UID: 1000 2021-02-20T14:39:11.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:39:16.388 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:39:16.389 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:39:16.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:39:16.390 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:39:16.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:39:16.390 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:39:16.393 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:39:16.393 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:40:07.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8112350776392754) > weight (0.25) 2021-02-20T14:40:07.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38336356868661814) > weight (0.25) 2021-02-20T14:40:07.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.717132737119321) > weight (0.25) 2021-02-20T14:40:07.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39346174486791974) > weight (0.25) 2021-02-20T14:40:07.522 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T14:41:09.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9566953591869001) > weight (0.25) 2021-02-20T14:41:09.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7055653733371342) > weight (0.25) 2021-02-20T14:41:09.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.38880829129765393) > weight (0.25) 2021-02-20T14:41:09.937 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:41:09.938 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:41:09.939 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:41:09.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:41:09.937+0000 7f5c2a2d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 65153) UID: 1000 2021-02-20T14:41:10.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:41:11.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:41:11.449+0000 7f0444dc9700 -1 rbd::mirror::ImageReplayer: 0x55f8ee443680 [3/c21d02f7-4a75-40cc-89e6-32ad02b9d7d4] start: already running 2021-02-20T14:41:11.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:41:11.449+0000 7f0444dc9700 -1 rbd::mirror::ImageReplayer: 0x55f8ee443900 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T14:41:14.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4027122060783248) > weight (0.25) 2021-02-20T14:41:14.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9310908834885747) > weight (0.25) 2021-02-20T14:41:14.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3917175127993364) > weight (0.25) 2021-02-20T14:41:14.523 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T14:41:14.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T14:41:14.524 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T14:41:14.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T14:41:14.525+0000 7f6ff650f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66408) UID: 1000 2021-02-20T14:41:14.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:41:15.939 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:41:15.940 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:41:15.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:41:15.940 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:41:15.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:41:15.941 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:41:15.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:41:15.946 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T14:41:21.525 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T14:41:21.526 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:41:21.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:41:21.527 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T14:41:21.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T14:41:21.528 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T14:41:21.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T14:41:21.533 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T14:41:25.168 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:41:25.164+0000 7f508e15e700 -1 reset not still connected to 0x55ac846e9c20 2021-02-20T14:41:25.169 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:41:25.164+0000 7f508e15e700 -1 reset not still connected to 0x55ac846e9cf0 2021-02-20T14:41:25.169 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:41:25.164+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e21c70 2021-02-20T14:41:25.169 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:41:25.164+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e222f0 2021-02-20T14:42:29.642 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:42:29.635+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8e29520 2021-02-20T14:42:29.643 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:42:29.635+0000 7f803c4bb700 -1 reset not still connected to 0x55acc13367d0 2021-02-20T14:42:29.644 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:42:29.639+0000 7f803c4bb700 -1 reset not still connected to 0x55accab01ba0 2021-02-20T14:42:29.644 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:42:29.639+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896d380 2021-02-20T14:42:29.644 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:42:29.639+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896dee0 2021-02-20T14:42:37.533 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.317960701153442) > weight (0.25) 2021-02-20T14:42:37.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.33476701324881775) > weight (0.25) 2021-02-20T14:42:37.534 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:42:37.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:42:37.535 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:42:37.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T14:42:37.535+0000 7f5cd2514700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 65730) UID: 1000 2021-02-20T14:42:37.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:42:43.536 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:42:43.543 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:42:43.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:42:43.544 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:42:43.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:42:43.545 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:42:43.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:42:43.548 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T14:43:07.947 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8337077040481767) > weight (0.25) 2021-02-20T14:43:07.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8205172642538162) > weight (0.25) 2021-02-20T14:43:07.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32459139990184926) > weight (0.25) 2021-02-20T14:43:07.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8015618939591547) > weight (0.25) 2021-02-20T14:43:07.957 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T14:44:28.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35385194696583033) > weight (0.25) 2021-02-20T14:44:28.551 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.33896258995599504) > weight (0.25) 2021-02-20T14:44:28.551 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8036846470899547) > weight (0.25) 2021-02-20T14:44:28.551 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8117290958531287) > weight (0.25) 2021-02-20T14:44:28.551 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T14:45:04.957 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:45:04.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:45:04.959 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:45:04.960 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:45:04.956+0000 7f04455ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66499) UID: 1000 2021-02-20T14:45:05.360 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:45:09.959 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:45:09.960 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:45:09.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:45:09.960 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:45:09.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:45:09.961 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:45:09.963 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:45:09.964 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T14:45:39.552 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:45:39.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:45:39.554 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T14:45:39.554 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:45:39.554 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:45:39.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:45:39.551+0000 7f63936ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65923) UID: 1000 2021-02-20T14:45:39.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:45:45.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:45:45.656 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:45:45.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:45:45.657 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:45:45.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:45:45.661 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T14:46:40.965 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:46:40.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:46:40.966 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:46:40.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:46:40.966+0000 7f2626e85700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67055) UID: 1000 2021-02-20T14:46:41.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:46:46.967 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:46:46.968 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:46:46.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:46:46.969 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:46:46.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:46:46.969 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:46:46.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:46:46.973 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T14:46:49.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7328873524091917) > weight (0.25) 2021-02-20T14:46:49.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4659483086901629) > weight (0.25) 2021-02-20T14:46:49.663 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:46:49.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:46:49.663 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:46:49.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T14:46:49.661+0000 7f002d31e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 66886) UID: 1000 2021-02-20T14:46:50.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:46:52.664 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:46:52.664 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:46:52.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:46:52.665 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:46:52.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:46:52.666 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:46:52.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:46:52.669 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T14:48:16.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3977531170829459) > weight (0.25) 2021-02-20T14:48:16.985 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9836142004687243) > weight (0.25) 2021-02-20T14:48:16.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7010805798971825) > weight (0.25) 2021-02-20T14:48:16.986 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T14:48:16.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T14:48:16.987 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T14:48:16.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T14:48:16.984+0000 7f2ffaf5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 66694) UID: 1000 2021-02-20T14:48:17.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:48:17.988 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T14:48:17.988 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:48:17.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:48:17.989 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T14:48:17.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T14:48:17.990 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T14:48:17.992 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T14:48:17.993 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:48:45.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9812540677376803) > weight (0.25) 2021-02-20T14:48:45.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9276924421575178) > weight (0.25) 2021-02-20T14:48:45.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8508831170993604) > weight (0.25) 2021-02-20T14:48:45.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8132171613401405) > weight (0.25) 2021-02-20T14:48:45.675 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T14:48:48.150 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:48:48.143+0000 7f803c4bb700 -1 reset not still connected to 0x55acbadbfc20 2021-02-20T14:50:00.994 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33698850776242395) > weight (0.25) 2021-02-20T14:50:01.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.399746877739521) > weight (0.25) 2021-02-20T14:50:01.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.29890259135459496) > weight (0.25) 2021-02-20T14:50:01.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7297729756034825) > weight (0.25) 2021-02-20T14:50:01.004 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T14:50:04.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5482142720761095) > weight (0.25) 2021-02-20T14:50:04.683 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45259357988211535) > weight (0.25) 2021-02-20T14:50:04.684 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T14:50:04.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T14:50:04.685 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:50:04.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T14:50:04.681+0000 7f75ef77a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 67337) UID: 1000 2021-02-20T14:50:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:50:09.686 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T14:50:09.692 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:50:09.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:50:09.693 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T14:50:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T14:50:09.694 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T14:50:09.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T14:50:09.698 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:50:33.244 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:50:33.235+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3a340 2021-02-20T14:50:33.440 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:50:33.235+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3c8a0 2021-02-20T14:50:33.441 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:50:33.235+0000 7f508e15e700 -1 reset not still connected to 0x55ac87272680 2021-02-20T14:50:33.441 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T14:50:33.235+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e22cb0 2021-02-20T14:51:42.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5489966626159993) > weight (0.25) 2021-02-20T14:51:42.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43440725150343873) > weight (0.25) 2021-02-20T14:51:42.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6662679719029456) > weight (0.25) 2021-02-20T14:51:42.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9640479775376695) > weight (0.25) 2021-02-20T14:51:42.006 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T14:51:59.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6181220635776048) > weight (0.25) 2021-02-20T14:51:59.700 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T14:51:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T14:51:59.701 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:51:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T14:51:59.699+0000 7fed8a649700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66081) UID: 1000 2021-02-20T14:52:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:52:05.702 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T14:52:05.702 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:52:05.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:52:05.703 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T14:52:05.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T14:52:05.704 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T14:52:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T14:52:05.707 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T14:53:23.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5067563459675053) > weight (0.25) 2021-02-20T14:53:23.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37486891358800223) > weight (0.25) 2021-02-20T14:53:23.009 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T14:53:23.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T14:53:23.010 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T14:53:23.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T14:53:23.009+0000 7fe7ffba4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 65638) UID: 1000 2021-02-20T14:53:23.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:53:26.012 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T14:53:26.012 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:53:26.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:53:26.013 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T14:53:26.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T14:53:26.014 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T14:53:26.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T14:53:26.016 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T14:53:42.708 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:53:42.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:53:42.709 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:53:42.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:53:42.704+0000 7f7766e7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67146) UID: 1000 2021-02-20T14:53:43.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:53:47.711 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:53:47.712 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:53:47.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:53:47.713 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:53:47.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:53:47.714 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:53:47.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:53:47.718 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T14:54:12.472 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.464+0000 7f803c4bb700 -1 reset not still connected to 0x55acb2f17fb0 2021-02-20T14:54:12.473 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.468+0000 7f803c4bb700 -1 reset not still connected to 0x55acb31a16c0 2021-02-20T14:54:12.474 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.468+0000 7f803c4bb700 -1 reset not still connected to 0x55acb88eca90 2021-02-20T14:54:12.474 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.468+0000 7f803c4bb700 -1 reset not still connected to 0x55accab00f70 2021-02-20T14:54:12.474 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.468+0000 7f803c4bb700 -1 reset not still connected to 0x55acdacf52b0 2021-02-20T14:54:12.475 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T14:54:12.468+0000 7f803c4bb700 -1 reset not still connected to 0x55acea4a0560 2021-02-20T14:54:30.017 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T14:54:30.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T14:54:30.018 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T14:54:30.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T14:54:30.015+0000 7f146c20f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67246) UID: 1000 2021-02-20T14:54:30.420 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:54:34.019 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T14:54:34.020 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:54:34.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:54:34.021 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T14:54:34.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T14:54:34.021 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T14:54:34.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T14:54:34.024 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T14:55:16.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6890790721965729) > weight (0.25) 2021-02-20T14:55:16.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6119989533636039) > weight (0.25) 2021-02-20T14:55:16.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.39877713614642074) > weight (0.25) 2021-02-20T14:55:16.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3702878883645475) > weight (0.25) 2021-02-20T14:55:16.721 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T14:56:17.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6156257538337822) > weight (0.25) 2021-02-20T14:56:17.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8067118707054516) > weight (0.25) 2021-02-20T14:56:17.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8215583307248995) > weight (0.25) 2021-02-20T14:56:17.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8592353802613554) > weight (0.25) 2021-02-20T14:56:17.027 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-02-20T14:57:06.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6043613004325624) > weight (0.25) 2021-02-20T14:57:06.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43689093226128295) > weight (0.25) 2021-02-20T14:57:06.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2821820124597426) > weight (0.25) 2021-02-20T14:57:06.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7683085548775035) > weight (0.25) 2021-02-20T14:57:06.724 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T14:57:59.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6046219262318991) > weight (0.25) 2021-02-20T14:57:59.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9370456480794408) > weight (0.25) 2021-02-20T14:57:59.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6306254676122051) > weight (0.25) 2021-02-20T14:57:59.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.46030086145969684) > weight (0.25) 2021-02-20T14:57:59.032 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T14:58:45.725 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T14:58:45.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T14:58:45.726 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T14:58:45.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T14:58:45.726+0000 7f6d80593700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67822) UID: 1000 2021-02-20T14:58:46.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:58:51.727 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T14:58:51.728 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:58:51.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:58:51.729 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T14:58:51.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T14:58:51.729 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T14:58:51.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T14:58:51.733 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T14:59:39.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4465946971581841) > weight (0.25) 2021-02-20T14:59:39.043 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T14:59:39.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T14:59:39.045 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T14:59:39.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T14:59:39.045+0000 7f1c943d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 66601) UID: 1000 2021-02-20T14:59:39.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T14:59:44.046 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T14:59:44.047 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T14:59:44.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T14:59:44.048 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T14:59:44.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T14:59:44.048 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T14:59:44.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T14:59:44.051 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:00:48.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9318421985766779) > weight (0.25) 2021-02-20T15:00:48.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7102128498699265) > weight (0.25) 2021-02-20T15:00:48.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6524420879986819) > weight (0.25) 2021-02-20T15:00:48.745 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:00:48.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:00:48.747 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:00:48.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:00:48.743+0000 7f28e2ed0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66785) UID: 1000 2021-02-20T15:00:49.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:00:50.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.986749033772287) > weight (0.25) 2021-02-20T15:00:50.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5872410123238894) > weight (0.25) 2021-02-20T15:00:50.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.855785197956907) > weight (0.25) 2021-02-20T15:00:50.060 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5259738065447236) > weight (0.25) 2021-02-20T15:00:50.060 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:00:57.749 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:00:57.749 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:00:57.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:00:57.751 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:00:57.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:00:57.752 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:00:57.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:00:57.760 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T15:02:47.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4704618652128221) > weight (0.25) 2021-02-20T15:02:47.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8386068091733576) > weight (0.25) 2021-02-20T15:02:47.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6974465850223585) > weight (0.25) 2021-02-20T15:02:47.064 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:02:47.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:02:47.066 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:02:47.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:02:47.061+0000 7f6ebf04f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 67438) UID: 1000 2021-02-20T15:02:47.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:02:47.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33240518908399264) > weight (0.25) 2021-02-20T15:02:47.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49058263886925224) > weight (0.25) 2021-02-20T15:02:47.765 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:02:47.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:02:47.765 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:02:47.766 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:02:47.766 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:02:47.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:02:47.761+0000 7f3bfb689700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 67530) UID: 1000 2021-02-20T15:02:48.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:02:53.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:02:53.868 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:02:53.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:02:53.869 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:02:53.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:02:53.872 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:02:54.068 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:02:54.069 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:02:54.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:02:54.070 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:02:54.070 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:02:54.070 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:02:54.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:02:54.073 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T15:03:59.075 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:03:59.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:03:59.077 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:03:59.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:03:59.071+0000 7ff7444e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67922) UID: 1000 2021-02-20T15:03:59.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:04:03.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:04:03.078 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:04:03.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:04:03.079 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:04:03.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:04:03.080 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:04:03.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:04:03.083 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T15:04:33.873 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28274311060495383) > weight (0.25) 2021-02-20T15:04:33.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8508176423033166) > weight (0.25) 2021-02-20T15:04:33.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9638817256581038) > weight (0.25) 2021-02-20T15:04:33.874 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:04:33.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:04:33.875 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:04:33.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:04:33.874+0000 7fe86340a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 68210) UID: 1000 2021-02-20T15:04:34.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:04:41.877 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:04:41.877 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:04:41.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:04:41.878 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:04:41.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:04:41.879 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:04:41.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:04:41.882 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:05:19.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8509037866356368) > weight (0.25) 2021-02-20T15:05:19.084 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:05:19.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:05:19.085 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:05:19.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:05:19.085+0000 7f61d9438700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 68118) UID: 1000 2021-02-20T15:05:19.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:05:20.086 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:05:20.087 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:05:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:05:20.088 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:05:20.089 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:05:20.090 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:05:20.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:05:20.096 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T15:06:07.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6328201189667799) > weight (0.25) 2021-02-20T15:06:07.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5033009143329776) > weight (0.25) 2021-02-20T15:06:07.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9709865409423526) > weight (0.25) 2021-02-20T15:06:07.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.73048823854889) > weight (0.25) 2021-02-20T15:06:07.898 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T15:06:50.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5758540881074501) > weight (0.25) 2021-02-20T15:06:50.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5714036247851184) > weight (0.25) 2021-02-20T15:06:50.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9426022001231352) > weight (0.25) 2021-02-20T15:06:50.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.64216965020056) > weight (0.25) 2021-02-20T15:06:50.106 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T15:07:33.899 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:07:33.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:07:33.900 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:07:33.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:07:33.899+0000 7f9eeb6ea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68018) UID: 1000 2021-02-20T15:07:34.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:07:38.901 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:07:38.902 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:07:38.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:07:38.903 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:07:38.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:07:38.904 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:07:38.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:07:38.907 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:08:23.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2619619927185365) > weight (0.25) 2021-02-20T15:08:23.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7965937019587253) > weight (0.25) 2021-02-20T15:08:23.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3611583997319284) > weight (0.25) 2021-02-20T15:08:23.129 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:08:23.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:08:23.130 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:08:23.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:08:23.129+0000 7efe8b0d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 68367) UID: 1000 2021-02-20T15:08:23.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:08:27.131 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:08:27.131 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:08:27.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:08:27.132 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:08:27.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:08:27.133 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:08:27.136 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:08:27.136 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:09:30.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6629080746527543) > weight (0.25) 2021-02-20T15:09:30.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5446276727140708) > weight (0.25) 2021-02-20T15:09:30.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5972989784451511) > weight (0.25) 2021-02-20T15:09:30.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8036346497546301) > weight (0.25) 2021-02-20T15:09:30.920 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T15:10:19.137 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:10:19.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:10:19.151 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:10:19.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:10:19.147+0000 7fee69739700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68501) UID: 1000 2021-02-20T15:10:19.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:10:27.152 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:10:27.160 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:10:27.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:10:27.161 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:10:27.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:10:27.162 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:10:27.164 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:10:27.165 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:10:59.921 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:10:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:10:59.930 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:10:59.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:10:59.930+0000 7f0161abb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68786) UID: 1000 2021-02-20T15:11:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:11:00.932 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:11:00.932 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:11:00.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:11:00.933 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:11:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:11:00.934 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:11:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:11:00.938 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T15:12:15.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8170361357416379) > weight (0.25) 2021-02-20T15:12:15.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7335187288757603) > weight (0.25) 2021-02-20T15:12:15.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5593824963460032) > weight (0.25) 2021-02-20T15:12:15.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.28967600987660835) > weight (0.25) 2021-02-20T15:12:15.168 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:12:55.940 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5311121938778538) > weight (0.25) 2021-02-20T15:12:55.941 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8206390095466328) > weight (0.25) 2021-02-20T15:12:55.941 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:12:55.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:12:55.942 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:12:55.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:12:55.940+0000 7f6d6959b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 68310) UID: 1000 2021-02-20T15:12:56.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:12:57.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:12:57.944 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:12:57.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:12:57.945 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:12:57.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:12:57.946 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:12:57.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:12:57.949 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T15:14:03.169 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:14:03.170 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:14:03.170 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:14:03.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:14:03.170+0000 7f2ef536a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68979) UID: 1000 2021-02-20T15:14:03.572 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:14:05.171 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:14:05.171 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:14:05.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:14:05.172 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:14:05.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:14:05.173 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:14:05.176 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:14:05.176 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T15:14:22.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7064556611864587) > weight (0.25) 2021-02-20T15:14:22.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9701597346422792) > weight (0.25) 2021-02-20T15:14:22.951 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3356311260269589) > weight (0.25) 2021-02-20T15:14:22.951 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:14:22.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:14:22.952 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:14:22.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:14:22.950+0000 7fd76138f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 68593) UID: 1000 2021-02-20T15:14:23.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:14:29.953 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:14:29.954 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:14:29.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:14:29.955 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:14:29.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:14:29.956 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:14:29.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:14:29.960 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T15:14:51.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T15:14:51.737+0000 7f1290318700 -1 rbd::mirror::ImageReplayer: 0x55908072d180 [3/c5428c9a-3b56-4be4-96e7-515eb901b226] start: already running 2021-02-20T15:15:29.178 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:15:29.179 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:15:29.179 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:15:29.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:15:29.180+0000 7f3a77346700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69269) UID: 1000 2021-02-20T15:15:29.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:15:37.181 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:15:37.181 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:15:37.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:15:37.182 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:15:37.182 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:15:37.183 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:15:37.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:15:37.187 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T15:15:43.961 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:15:43.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:15:43.963 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:15:43.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:15:43.960+0000 7f13d6cef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69070) UID: 1000 2021-02-20T15:15:44.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:15:49.964 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:15:49.965 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:15:49.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:15:49.966 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:15:49.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:15:49.966 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:15:49.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:15:49.970 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T15:16:58.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7238264825729716) > weight (0.25) 2021-02-20T15:16:58.973 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9102680367306532) > weight (0.25) 2021-02-20T15:16:58.973 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:16:58.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:16:58.975 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:16:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:16:58.974+0000 7f42ab63d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 69169) UID: 1000 2021-02-20T15:16:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:17:06.976 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:17:06.976 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:17:06.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:17:06.977 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:17:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:17:06.978 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:17:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:17:06.981 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T15:17:08.188 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4517805038977515) > weight (0.25) 2021-02-20T15:17:08.188 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:17:08.189 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:17:08.189 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:17:08.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:17:08.186+0000 7f16fd00a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 68695) UID: 1000 2021-02-20T15:17:08.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:17:13.190 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:17:13.191 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:17:13.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:17:13.192 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:17:13.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:17:13.192 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:17:13.195 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:17:13.195 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:17:36.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:17:36.994+0000 7f4dc009e700 -1 rbd::mirror::ImageReplayer: 0x562f0c916580 [3/960fd008-da78-4ef8-82ac-65f761a41a17] start: already running 2021-02-20T15:18:53.197 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:18:53.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:18:53.199 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:18:53.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:18:53.196+0000 7f6d00274700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69463) UID: 1000 2021-02-20T15:18:53.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:18:54.200 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:18:54.201 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:18:54.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:18:54.202 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:18:54.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:18:54.203 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:18:54.205 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:18:54.206 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T15:19:00.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6446649119907341) > weight (0.25) 2021-02-20T15:19:00.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.25722182949563366) > weight (0.25) 2021-02-20T15:19:00.982 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8559443714592904) > weight (0.25) 2021-02-20T15:19:00.983 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6667460390200479) > weight (0.25) 2021-02-20T15:19:00.983 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:19:26.284 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:19:26.283+0000 7f508e15e700 -1 reset not still connected to 0x55ac8295bba0 2021-02-20T15:19:26.285 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:19:26.283+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3b6c0 2021-02-20T15:20:26.207 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:20:26.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:20:26.218 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:20:26.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:20:26.218+0000 7fb1f4f19700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69848) UID: 1000 2021-02-20T15:20:26.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:20:30.219 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:20:30.219 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:20:30.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:20:30.220 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:20:30.220 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:20:30.221 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:20:30.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:20:30.224 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T15:20:57.984 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:20:57.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:20:57.994 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:20:57.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:20:57.989+0000 7f6df310d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69554) UID: 1000 2021-02-20T15:20:58.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:21:05.995 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:21:05.996 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:21:05.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:21:05.996 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:21:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:21:05.997 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:21:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:21:06.000 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T15:22:05.225 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6155718614548249) > weight (0.25) 2021-02-20T15:22:05.235 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:22:05.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:22:05.237 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:22:05.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:22:05.232+0000 7fc26a920700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69756) UID: 1000 2021-02-20T15:22:05.638 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:22:08.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:22:08.238 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:22:08.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:22:08.239 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:22:08.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:22:08.240 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:22:08.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:22:08.243 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T15:22:15.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5104264693572604) > weight (0.25) 2021-02-20T15:22:15.005 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3972718538358725) > weight (0.25) 2021-02-20T15:22:15.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8779306171375941) > weight (0.25) 2021-02-20T15:22:15.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.43068236244516667) > weight (0.25) 2021-02-20T15:22:15.006 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:23:10.244 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:23:10.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:23:10.252 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:23:10.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:23:10.250+0000 7f05b4e83700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69941) UID: 1000 2021-02-20T15:23:10.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:23:18.253 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:23:18.253 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:23:18.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:23:18.254 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:23:18.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:23:18.255 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:23:18.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:23:18.258 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:23:25.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.37032437808838126) > weight (0.25) 2021-02-20T15:23:25.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3183267725142459) > weight (0.25) 2021-02-20T15:23:25.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4618728915063335) > weight (0.25) 2021-02-20T15:23:25.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4691736736538702) > weight (0.25) 2021-02-20T15:23:25.009 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T15:24:30.259 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5986074926242972) > weight (0.25) 2021-02-20T15:24:30.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4987153118294345) > weight (0.25) 2021-02-20T15:24:30.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39737006645586026) > weight (0.25) 2021-02-20T15:24:30.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5458935874426435) > weight (0.25) 2021-02-20T15:24:30.260 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T15:24:40.011 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.743310490050581) > weight (0.25) 2021-02-20T15:24:40.011 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7126417188768297) > weight (0.25) 2021-02-20T15:24:40.011 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7530037985996718) > weight (0.25) 2021-02-20T15:24:40.012 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.34770398316095663) > weight (0.25) 2021-02-20T15:24:40.012 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T15:25:36.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7763053895528657) > weight (0.25) 2021-02-20T15:25:36.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7649383672178716) > weight (0.25) 2021-02-20T15:25:36.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8767222236189011) > weight (0.25) 2021-02-20T15:25:36.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.35880899502964214) > weight (0.25) 2021-02-20T15:25:36.262 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:25:59.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5059415820065201) > weight (0.25) 2021-02-20T15:25:59.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7214629422353492) > weight (0.25) 2021-02-20T15:25:59.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5872714357844927) > weight (0.25) 2021-02-20T15:25:59.014 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:25:59.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:25:59.016 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:25:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:25:59.015+0000 7f4c03d79700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 69360) UID: 1000 2021-02-20T15:25:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:26:01.017 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:26:01.017 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:26:01.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:26:01.019 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:26:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:26:01.019 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:26:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:26:01.023 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T15:27:10.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3360699467738618) > weight (0.25) 2021-02-20T15:27:10.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4667702637990364) > weight (0.25) 2021-02-20T15:27:10.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.48545171663469344) > weight (0.25) 2021-02-20T15:27:10.264 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:27:10.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:27:10.265 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:27:10.265 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:27:10.266 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:27:10.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:27:10.265+0000 7f4dc089f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 68886) UID: 1000 2021-02-20T15:27:10.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:27:16.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:27:16.368 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:27:16.369 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:27:16.369 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:27:16.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:27:16.372 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T15:27:24.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2642578315639321) > weight (0.25) 2021-02-20T15:27:24.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4652573545221379) > weight (0.25) 2021-02-20T15:27:24.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9761381560452519) > weight (0.25) 2021-02-20T15:27:24.024 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:27:24.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:27:24.025 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:27:24.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:27:24.025+0000 7f7ea2867700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 70319) UID: 1000 2021-02-20T15:27:24.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:27:30.026 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:27:30.027 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:27:30.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:27:30.028 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:27:30.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:27:30.028 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:27:30.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:27:30.032 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T15:28:38.373 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8004067488141083) > weight (0.25) 2021-02-20T15:28:38.374 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:28:38.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:28:38.375 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:28:38.376 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:28:38.375+0000 7f403b173700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70132) UID: 1000 2021-02-20T15:28:38.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:28:39.376 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:28:39.377 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:28:39.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:28:39.378 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:28:39.378 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:28:39.378 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:28:39.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:28:39.381 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T15:29:01.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3405696391435228) > weight (0.25) 2021-02-20T15:29:01.033 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8765638913941262) > weight (0.25) 2021-02-20T15:29:01.033 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:29:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:29:01.034 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:29:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:29:01.035+0000 7fe1e813f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 69657) UID: 1000 2021-02-20T15:29:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:29:03.035 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:29:03.036 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:29:03.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:29:03.037 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:29:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:29:03.037 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:29:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:29:03.040 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T15:29:41.382 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6366430415257307) > weight (0.25) 2021-02-20T15:29:41.383 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7760817178221376) > weight (0.25) 2021-02-20T15:29:41.383 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8195036789164075) > weight (0.25) 2021-02-20T15:29:41.383 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.43204315563967377) > weight (0.25) 2021-02-20T15:29:41.383 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T15:30:47.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5717632504897492) > weight (0.25) 2021-02-20T15:30:47.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6400374378409331) > weight (0.25) 2021-02-20T15:30:47.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4402832114142561) > weight (0.25) 2021-02-20T15:30:47.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8872153903038583) > weight (0.25) 2021-02-20T15:30:47.043 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T15:31:32.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7231855427614862) > weight (0.25) 2021-02-20T15:31:32.385 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:31:32.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:31:32.386 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:31:32.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:31:32.383+0000 7f9c40723700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70609) UID: 1000 2021-02-20T15:31:32.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:31:38.387 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:31:38.387 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:31:38.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:31:38.388 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:31:38.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:31:38.389 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:31:38.392 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:31:38.392 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T15:32:42.044 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:32:42.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:32:42.046 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:32:42.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:32:42.046+0000 7f73fc0d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70033) UID: 1000 2021-02-20T15:32:42.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:32:43.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27438141677672967) > weight (0.25) 2021-02-20T15:32:43.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9290971315184239) > weight (0.25) 2021-02-20T15:32:43.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7084693705877418) > weight (0.25) 2021-02-20T15:32:43.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9794338460993157) > weight (0.25) 2021-02-20T15:32:43.395 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:32:46.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:32:46.047 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:32:46.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:32:46.048 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:32:46.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:32:46.049 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:32:46.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:32:46.052 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:32:51.633 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T15:32:51.630+0000 7f803c4bb700 -1 reset not still connected to 0x55acb31a3670 2021-02-20T15:32:51.633 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T15:32:51.630+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896d860 2021-02-20T15:32:51.634 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T15:32:51.630+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896f810 2021-02-20T15:34:20.054 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:34:20.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:34:20.056 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:34:20.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:34:20.056+0000 7f94eff9c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70893) UID: 1000 2021-02-20T15:34:20.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:34:28.057 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:34:28.058 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:34:28.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:34:28.059 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:34:28.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:34:28.059 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:34:28.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:34:28.062 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:34:31.734 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:34:31.730+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e21520 2021-02-20T15:34:35.396 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:34:35.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:34:35.397 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:34:35.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:34:35.391+0000 7f8b920fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70223) UID: 1000 2021-02-20T15:34:35.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:34:39.398 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:34:39.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:34:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:34:39.399 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:34:39.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:34:39.400 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:34:39.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:34:39.403 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T15:34:48.795 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:34:48.789+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d947930 2021-02-20T15:34:48.799 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:34:48.793+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d948560 2021-02-20T15:34:48.799 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:34:48.793+0000 7f508e15e700 -1 reset not still connected to 0x55ac7d948e50 2021-02-20T15:34:48.800 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:34:48.793+0000 7f508e15e700 -1 reset not still connected to 0x55ac88ef6f20 2021-02-20T15:35:43.404 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39258082773986525) > weight (0.25) 2021-02-20T15:35:43.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6703385176466683) > weight (0.25) 2021-02-20T15:35:43.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.35823682297594095) > weight (0.25) 2021-02-20T15:35:43.406 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:35:43.406 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:35:43.407 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:35:43.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:35:43.406+0000 7f701feca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 70419) UID: 1000 2021-02-20T15:35:43.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:35:46.407 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:35:46.408 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:35:46.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:35:46.409 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:35:46.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:35:46.409 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:35:46.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:35:46.412 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T15:35:56.290 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:35:56.284+0000 7f508e15e700 -1 reset not still connected to 0x55ac8295bba0 2021-02-20T15:35:56.290 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:35:56.284+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3b6c0 2021-02-20T15:35:56.294 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T15:35:56.288+0000 7f508e15e700 -1 reset not still connected to 0x55ac85c3cb10 2021-02-20T15:36:25.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.860882469137173) > weight (0.25) 2021-02-20T15:36:25.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5725791755766803) > weight (0.25) 2021-02-20T15:36:25.064 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:36:25.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:36:25.065 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:36:25.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:36:25.061+0000 7fb4f1ae4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 70700) UID: 1000 2021-02-20T15:36:25.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:36:33.066 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:36:33.067 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:36:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:36:33.068 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:36:33.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:36:33.069 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:36:33.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:36:33.073 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T15:36:58.191 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T15:36:58.188+0000 7fc273bc1700 -1 reset not still connected to 0x56119260bfb0 2021-02-20T15:37:08.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35364434866383854) > weight (0.25) 2021-02-20T15:37:08.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7726098188362636) > weight (0.25) 2021-02-20T15:37:08.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33400885771800837) > weight (0.25) 2021-02-20T15:37:08.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4014198384814518) > weight (0.25) 2021-02-20T15:37:08.415 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T15:38:16.416 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:38:16.417 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:38:16.418 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:38:16.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:38:16.415+0000 7f252056a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71091) UID: 1000 2021-02-20T15:38:16.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:38:17.419 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:38:17.420 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:38:17.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:38:17.421 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:38:17.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:38:17.422 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:38:17.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:38:17.425 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T15:38:18.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3435645423044462) > weight (0.25) 2021-02-20T15:38:18.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7499581112985143) > weight (0.25) 2021-02-20T15:38:18.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7684317544738042) > weight (0.25) 2021-02-20T15:38:18.075 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T15:38:18.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T15:38:18.076 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:38:18.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T15:38:18.075+0000 7fa099f88700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 70510) UID: 1000 2021-02-20T15:38:18.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:38:22.076 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T15:38:22.077 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:38:22.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:38:22.077 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T15:38:22.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T15:38:22.078 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T15:38:22.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T15:38:22.081 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:39:24.426 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3019214612738441) > weight (0.25) 2021-02-20T15:39:24.427 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:39:24.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:39:24.428 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T15:39:24.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:39:24.425+0000 7f7f217f2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70801) UID: 1000 2021-02-20T15:39:24.830 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:39:32.428 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:39:32.429 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:39:32.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:39:32.430 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:39:32.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:39:32.430 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:39:32.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:39:32.433 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:40:10.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48328463883171024) > weight (0.25) 2021-02-20T15:40:10.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4507200529992481) > weight (0.25) 2021-02-20T15:40:10.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7864391191619623) > weight (0.25) 2021-02-20T15:40:10.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7123248118752883) > weight (0.25) 2021-02-20T15:40:10.084 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T15:41:21.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4151760170678408) > weight (0.25) 2021-02-20T15:41:21.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2946302859950991) > weight (0.25) 2021-02-20T15:41:21.435 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9460695519039541) > weight (0.25) 2021-02-20T15:41:21.436 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.45351961492444326) > weight (0.25) 2021-02-20T15:41:21.436 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T15:42:00.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3862554199775876) > weight (0.25) 2021-02-20T15:42:00.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5447717323106358) > weight (0.25) 2021-02-20T15:42:00.086 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:42:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:42:00.088 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T15:42:00.088 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:42:00.088 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:42:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:42:00.086+0000 7f17bf18d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 71277) UID: 1000 2021-02-20T15:42:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:42:06.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:42:06.536 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:42:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:42:06.537 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:42:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:42:06.540 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T15:43:11.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4414834625078218) > weight (0.25) 2021-02-20T15:43:11.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7251809709550734) > weight (0.25) 2021-02-20T15:43:11.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9278948223434641) > weight (0.25) 2021-02-20T15:43:11.438 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:43:11.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:43:11.439 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:43:11.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:43:11.436+0000 7fbcb2632700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71186) UID: 1000 2021-02-20T15:43:11.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:43:15.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6223557231552184) > weight (0.25) 2021-02-20T15:43:15.542 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:43:15.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:43:15.543 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T15:43:15.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T15:43:15.540+0000 7f129131a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 67630) UID: 1000 2021-02-20T15:43:15.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:43:18.440 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:43:18.440 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:43:18.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:43:18.441 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:43:18.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:43:18.442 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:43:18.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:43:18.444 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T15:43:24.544 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:43:24.544 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:43:24.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:43:24.545 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:43:24.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:43:24.545 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:43:24.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:43:24.548 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T15:44:32.446 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8856901494978394) > weight (0.25) 2021-02-20T15:44:32.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8424991784423006) > weight (0.25) 2021-02-20T15:44:32.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2997350759501112) > weight (0.25) 2021-02-20T15:44:32.456 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:44:32.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:44:32.457 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T15:44:32.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:44:32.455+0000 7f62237be700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71761) UID: 1000 2021-02-20T15:44:32.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48983766352111724) > weight (0.25) 2021-02-20T15:44:32.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3807691928288093) > weight (0.25) 2021-02-20T15:44:32.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9567528963319211) > weight (0.25) 2021-02-20T15:44:32.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9933350682928166) > weight (0.25) 2021-02-20T15:44:32.558 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T15:44:32.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:44:39.458 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:44:39.459 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:44:39.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:44:39.460 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:44:39.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:44:39.461 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:44:39.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:44:39.464 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T15:44:43.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:44:43.302+0000 7f0fac3ed700 -1 rbd::mirror::ImageReplayer: 0x56472aedfb80 [3/eb5ee405-f45d-4109-b887-4acc3932b673] start: already running 2021-02-20T15:45:33.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6840892570981609) > weight (0.25) 2021-02-20T15:45:33.570 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43923764177840474) > weight (0.25) 2021-02-20T15:45:33.570 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46764993324211857) > weight (0.25) 2021-02-20T15:45:33.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3089079297439147) > weight (0.25) 2021-02-20T15:45:33.571 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T15:45:48.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31761365714556555) > weight (0.25) 2021-02-20T15:45:48.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3747712195656159) > weight (0.25) 2021-02-20T15:45:48.465 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30847974085070007) > weight (0.25) 2021-02-20T15:45:48.466 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:45:48.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:45:48.468 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:45:48.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:45:48.465+0000 7f0facbee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71951) UID: 1000 2021-02-20T15:45:48.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:45:53.469 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:45:53.475 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:45:53.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:45:53.476 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:45:53.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:45:53.477 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:45:53.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:45:53.480 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T15:46:46.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4208834650333997) > weight (0.25) 2021-02-20T15:46:46.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5364994934628102) > weight (0.25) 2021-02-20T15:46:46.573 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T15:46:46.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T15:46:46.574 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:46:46.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T15:46:46.568+0000 7fc90cc13700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 71661) UID: 1000 2021-02-20T15:46:46.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:46:50.574 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T15:46:50.575 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:46:50.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:46:50.576 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T15:46:50.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T15:46:50.576 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T15:46:50.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T15:46:50.579 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T15:47:33.481 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:47:33.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:47:33.483 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:47:33.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:47:33.483+0000 7f63195e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71378) UID: 1000 2021-02-20T15:47:33.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:47:34.484 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:47:34.485 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:47:34.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:47:34.486 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:47:34.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:47:34.487 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:47:34.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:47:34.490 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T15:48:42.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47663654190027827) > weight (0.25) 2021-02-20T15:48:42.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.741152663445523) > weight (0.25) 2021-02-20T15:48:42.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.38930367157769963) > weight (0.25) 2021-02-20T15:48:42.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49102882132894055) > weight (0.25) 2021-02-20T15:48:42.582 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T15:49:13.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9266439956942084) > weight (0.25) 2021-02-20T15:49:13.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7727215747018519) > weight (0.25) 2021-02-20T15:49:13.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.527081600909793) > weight (0.25) 2021-02-20T15:49:13.492 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7757698068048806) > weight (0.25) 2021-02-20T15:49:13.492 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T15:50:38.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8330451790019343) > weight (0.25) 2021-02-20T15:50:38.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5776068439046148) > weight (0.25) 2021-02-20T15:50:38.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5516611398896594) > weight (0.25) 2021-02-20T15:50:38.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5297605719842835) > weight (0.25) 2021-02-20T15:50:38.584 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T15:51:12.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.775396326992429) > weight (0.25) 2021-02-20T15:51:12.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3645689691236632) > weight (0.25) 2021-02-20T15:51:12.495 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7470346535803757) > weight (0.25) 2021-02-20T15:51:12.495 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T15:51:12.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T15:51:12.496 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:51:12.497 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T15:51:12.494+0000 7f8c0be73700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 72045) UID: 1000 2021-02-20T15:51:12.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:51:14.497 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T15:51:14.497 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:51:14.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:51:14.498 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T15:51:14.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T15:51:14.499 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T15:51:14.503 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T15:51:14.503 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T15:51:41.585 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T15:51:41.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T15:51:41.586 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T15:51:41.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T15:51:41.585+0000 7f3759c28700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70992) UID: 1000 2021-02-20T15:51:41.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:51:47.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T15:51:47.588 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:51:47.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:51:47.588 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T15:51:47.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T15:51:47.589 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T15:51:47.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T15:51:47.592 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:52:33.505 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4069978605023836) > weight (0.25) 2021-02-20T15:52:33.506 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:52:33.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:52:33.507 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T15:52:33.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:52:33.504+0000 7fc81be1a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71568) UID: 1000 2021-02-20T15:52:33.909 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:52:36.508 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:52:36.509 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:52:36.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:52:36.510 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:52:36.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:52:36.510 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:52:36.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:52:36.513 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T15:52:57.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26127774467117826) > weight (0.25) 2021-02-20T15:52:57.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9543734530020249) > weight (0.25) 2021-02-20T15:52:57.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8048610630637638) > weight (0.25) 2021-02-20T15:52:57.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.25383660863884483) > weight (0.25) 2021-02-20T15:52:57.594 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T15:53:48.514 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:53:48.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:53:48.516 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:53:48.517 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:53:48.515+0000 7fc3c2629700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72236) UID: 1000 2021-02-20T15:53:48.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:53:53.517 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:53:53.518 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:53:53.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:53:53.518 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:53:53.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:53:53.519 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:53:53.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:53:53.522 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T15:53:57.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6776055982534445) > weight (0.25) 2021-02-20T15:53:57.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8274746799938445) > weight (0.25) 2021-02-20T15:53:57.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7054218932719376) > weight (0.25) 2021-02-20T15:53:57.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5797025432233757) > weight (0.25) 2021-02-20T15:53:57.596 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T15:55:03.524 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:55:03.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:55:03.526 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T15:55:03.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:55:03.525+0000 7f979cadf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72614) UID: 1000 2021-02-20T15:55:03.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:55:05.527 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:55:05.528 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:55:05.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:55:05.529 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:55:05.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:55:05.530 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:55:05.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:55:05.533 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T15:55:54.597 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6367215933940038) > weight (0.25) 2021-02-20T15:55:54.598 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T15:55:54.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T15:55:54.599 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T15:55:54.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T15:55:54.596+0000 7f876688b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 71852) UID: 1000 2021-02-20T15:55:55.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:55:55.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T15:55:55.600 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:55:55.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:55:55.601 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T15:55:55.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T15:55:55.602 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T15:55:55.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T15:55:55.605 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T15:56:13.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3941663498975029) > weight (0.25) 2021-02-20T15:56:13.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7628332270334061) > weight (0.25) 2021-02-20T15:56:13.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8072980800478884) > weight (0.25) 2021-02-20T15:56:13.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.80056186970583) > weight (0.25) 2021-02-20T15:56:13.535 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T15:57:38.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7338455535162111) > weight (0.25) 2021-02-20T15:57:38.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7367982688051296) > weight (0.25) 2021-02-20T15:57:38.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6004631942526395) > weight (0.25) 2021-02-20T15:57:38.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6391382545511818) > weight (0.25) 2021-02-20T15:57:38.608 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T15:58:02.537 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T15:58:02.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T15:58:02.539 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T15:58:02.539 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T15:58:02.537+0000 7f4c6f867700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72708) UID: 1000 2021-02-20T15:58:02.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:58:07.540 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T15:58:07.540 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:58:07.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:58:07.541 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T15:58:07.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T15:58:07.542 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T15:58:07.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T15:58:07.545 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T15:58:51.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7527440605887687) > weight (0.25) 2021-02-20T15:58:51.610 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3571434754324361) > weight (0.25) 2021-02-20T15:58:51.610 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7026159835604561) > weight (0.25) 2021-02-20T15:58:51.611 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3264570971579257) > weight (0.25) 2021-02-20T15:58:51.611 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T15:59:55.547 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5564994125414515) > weight (0.25) 2021-02-20T15:59:55.547 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T15:59:55.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T15:59:55.549 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T15:59:55.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T15:59:55.547+0000 7fdd54277700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72522) UID: 1000 2021-02-20T15:59:55.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T15:59:59.550 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T15:59:59.551 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T15:59:59.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T15:59:59.551 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T15:59:59.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T15:59:59.552 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T15:59:59.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T15:59:59.555 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T16:00:25.612 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7659730096134283) > weight (0.25) 2021-02-20T16:00:25.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9812766846475633) > weight (0.25) 2021-02-20T16:00:25.613 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:00:25.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:00:25.614 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:00:25.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T16:00:25.614+0000 7f9ebd736700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 72137) UID: 1000 2021-02-20T16:00:26.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:00:32.615 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:00:32.615 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:00:32.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:00:32.616 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:00:32.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:00:32.617 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:00:32.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:00:32.620 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T16:01:38.622 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4617015862992332) > weight (0.25) 2021-02-20T16:01:38.622 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6276000213923217) > weight (0.25) 2021-02-20T16:01:38.623 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2786320988479021) > weight (0.25) 2021-02-20T16:01:38.623 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T16:01:38.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T16:01:38.624 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:01:38.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T16:01:38.620+0000 7f409802b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 71469) UID: 1000 2021-02-20T16:01:39.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:01:39.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:01:39.900+0000 7f3a317ea700 -1 rbd::mirror::ImageReplayer: 0x559e195b7680 [3/4c4f4929-06c7-4203-81c6-a0c9560ab291] start: already running 2021-02-20T16:01:39.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:01:39.900+0000 7f3a317ea700 -1 rbd::mirror::ImageReplayer: 0x559e1d6fea00 [3/4d947326-1304-4f75-9cb5-1cc9895ee2c8] start: already running 2021-02-20T16:01:39.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:01:39.900+0000 7f3a317ea700 -1 rbd::mirror::ImageReplayer: 0x559e1d6ff900 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T16:01:39.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:01:39.900+0000 7f3a317ea700 -1 rbd::mirror::ImageReplayer: 0x559e195b7400 [3/9a469893-68d6-4d91-a371-3899fcc47454] start: already running 2021-02-20T16:01:43.624 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T16:01:43.625 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:01:43.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:01:43.626 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T16:01:43.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T16:01:43.628 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T16:01:43.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T16:01:43.631 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:01:52.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5338424155023367) > weight (0.25) 2021-02-20T16:01:52.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6472710292423668) > weight (0.25) 2021-02-20T16:01:52.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40888114314257873) > weight (0.25) 2021-02-20T16:01:52.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8649059564996595) > weight (0.25) 2021-02-20T16:01:52.558 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T16:02:14.769 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:02:14.764+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1337dc0 2021-02-20T16:02:56.559 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:02:56.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:02:56.560 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:02:56.561 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:02:56.559+0000 7f888f7b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72899) UID: 1000 2021-02-20T16:02:56.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:02:58.560 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:02:58.561 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:02:58.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:02:58.561 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:02:58.562 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:02:58.562 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:02:58.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:02:58.565 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T16:03:03.269 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T16:03:03.265+0000 7f62c7ffc700 -1 reset not still connected to 0x555b53c9d2b0 2021-02-20T16:03:03.269 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T16:03:03.265+0000 7f62c7ffc700 -1 reset not still connected to 0x555b53c9f740 2021-02-20T16:03:26.632 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:03:26.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:03:26.633 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:03:26.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:03:26.630+0000 7f8417756700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 72423) UID: 1000 2021-02-20T16:03:27.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:03:29.634 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:03:29.635 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:03:29.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:03:29.635 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:03:29.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:03:29.636 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:03:29.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:03:29.639 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:04:24.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7743822759211977) > weight (0.25) 2021-02-20T16:04:24.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.28291996051238866) > weight (0.25) 2021-02-20T16:04:24.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6376931070409421) > weight (0.25) 2021-02-20T16:04:24.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7787884501187589) > weight (0.25) 2021-02-20T16:04:24.567 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T16:05:05.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3325667918484837) > weight (0.25) 2021-02-20T16:05:05.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4671322258409426) > weight (0.25) 2021-02-20T16:05:05.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3167083005511331) > weight (0.25) 2021-02-20T16:05:05.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7339971669515756) > weight (0.25) 2021-02-20T16:05:05.642 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T16:06:03.568 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6276055720328545) > weight (0.25) 2021-02-20T16:06:03.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.822735520103168) > weight (0.25) 2021-02-20T16:06:03.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3493055295393629) > weight (0.25) 2021-02-20T16:06:03.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9577077795736911) > weight (0.25) 2021-02-20T16:06:03.569 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T16:06:55.643 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:06:55.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:06:55.645 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:06:55.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:06:55.641+0000 7fda51b90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73374) UID: 1000 2021-02-20T16:06:56.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:07:00.645 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:07:00.646 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:07:00.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:07:00.646 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:07:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:07:00.647 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:07:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:07:00.650 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T16:07:40.571 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:07:40.572 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:07:40.572 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:07:40.573 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:07:40.569+0000 7f368c250700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 73282) UID: 1000 2021-02-20T16:07:40.974 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:07:42.573 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:07:42.574 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:07:42.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:07:42.574 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:07:42.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:07:42.575 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:07:42.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:07:42.579 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T16:08:33.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9776008036766327) > weight (0.25) 2021-02-20T16:08:33.652 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:08:33.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:08:33.653 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:08:33.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:08:33.647+0000 7f3a327ec700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 72799) UID: 1000 2021-02-20T16:08:34.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:08:34.654 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:08:34.654 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:08:34.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:08:34.655 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:08:34.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:08:34.656 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:08:34.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:08:34.659 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:09:01.471 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:09:01.465+0000 7f508e15e700 -1 reset not still connected to 0x55ac8f710dd0 2021-02-20T16:09:01.471 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:09:01.465+0000 7f508e15e700 -1 reset not still connected to 0x55ac8f711520 2021-02-20T16:09:01.472 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:09:01.465+0000 7f508e15e700 -1 reset not still connected to 0x55ac8f711e10 2021-02-20T16:09:28.580 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.28136750623995677) > weight (0.25) 2021-02-20T16:09:28.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6850670385401998) > weight (0.25) 2021-02-20T16:09:28.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4154353186456109) > weight (0.25) 2021-02-20T16:09:28.581 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:09:28.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:09:28.582 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:09:28.583 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T16:09:28.578+0000 7f9aa0159700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 72331) UID: 1000 2021-02-20T16:09:28.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:09:37.584 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:09:37.584 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:09:37.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:09:37.586 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:09:37.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:09:37.587 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:09:37.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:09:37.591 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:09:51.660 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:09:51.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:09:51.661 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:09:51.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:09:51.658+0000 7fa4c0096700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73478) UID: 1000 2021-02-20T16:09:52.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:09:57.662 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:09:57.662 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:09:57.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:09:57.663 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:09:57.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:09:57.664 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:09:57.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:09:57.667 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:11:18.592 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9312766085267027) > weight (0.25) 2021-02-20T16:11:18.592 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:11:18.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:11:18.594 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:11:18.594 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:11:18.592+0000 7fa567037700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72992) UID: 1000 2021-02-20T16:11:18.995 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:11:23.595 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:11:23.595 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:11:23.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:11:23.596 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:11:23.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:11:23.597 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:11:23.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:11:23.601 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T16:11:53.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6476278035354062) > weight (0.25) 2021-02-20T16:11:53.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.610380242552086) > weight (0.25) 2021-02-20T16:11:53.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.532202655524556) > weight (0.25) 2021-02-20T16:11:53.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7370031132456213) > weight (0.25) 2021-02-20T16:11:53.671 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T16:12:53.859 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:12:53.851+0000 7f508e15e700 -1 reset not still connected to 0x55ac918043c0 2021-02-20T16:12:56.602 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7986137272536086) > weight (0.25) 2021-02-20T16:12:56.602 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.48965343984761245) > weight (0.25) 2021-02-20T16:12:56.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8531443888684702) > weight (0.25) 2021-02-20T16:12:56.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5570462055925834) > weight (0.25) 2021-02-20T16:12:56.603 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T16:13:42.852 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T16:13:42.846+0000 7f62c7ffc700 -1 reset not still connected to 0x555b42f7e8f0 2021-02-20T16:13:46.671 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40642615645372093) > weight (0.25) 2021-02-20T16:13:46.672 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:13:46.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:13:46.673 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:13:46.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:13:46.669+0000 7f61c98ec700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 73669) UID: 1000 2021-02-20T16:13:47.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:13:52.673 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:13:52.674 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:13:52.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:13:52.675 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:13:52.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:13:52.675 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:13:52.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:13:52.678 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T16:14:22.604 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:14:22.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:14:22.606 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:14:22.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:14:22.604+0000 7f1a499d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 73577) UID: 1000 2021-02-20T16:14:23.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:14:30.607 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:14:30.608 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:14:30.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:14:30.609 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:14:30.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:14:30.610 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:14:30.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:14:30.614 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T16:14:58.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8876827417847032) > weight (0.25) 2021-02-20T16:14:58.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43993073030779817) > weight (0.25) 2021-02-20T16:14:58.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9360236491120679) > weight (0.25) 2021-02-20T16:14:58.680 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6281441131439601) > weight (0.25) 2021-02-20T16:14:58.681 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T16:15:42.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38811335780559053) > weight (0.25) 2021-02-20T16:15:42.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6797406506717336) > weight (0.25) 2021-02-20T16:15:42.616 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:15:42.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:15:42.617 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:15:42.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T16:15:42.614+0000 7fc9b0847700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 67730) UID: 1000 2021-02-20T16:15:43.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:15:43.618 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:15:43.618 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:15:43.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:15:43.619 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:15:43.620 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:15:43.620 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:15:43.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:15:43.624 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:16:06.682 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:16:06.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:16:06.683 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:16:06.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:16:06.678+0000 7fd07433e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73860) UID: 1000 2021-02-20T16:16:07.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:16:12.684 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:16:12.684 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:16:12.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:16:12.685 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:16:12.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:16:12.686 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:16:12.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:16:12.689 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T16:16:37.073 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:16:37.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac862c9ba0 2021-02-20T16:16:37.074 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:16:37.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac88b1e9c0 2021-02-20T16:16:37.074 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:16:37.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac88d99810 2021-02-20T16:16:37.075 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:16:37.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac893f8ff0 2021-02-20T16:16:37.075 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:16:37.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac8f710ea0 2021-02-20T16:17:15.690 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:17:15.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:17:15.691 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:17:15.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:17:15.684+0000 7fc8cda7c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74338) UID: 1000 2021-02-20T16:17:16.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:17:22.693 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:17:22.693 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:17:22.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:17:22.694 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:17:22.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:17:22.695 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:17:22.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:17:22.698 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:17:26.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8278879914345828) > weight (0.25) 2021-02-20T16:17:26.626 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:17:26.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:17:26.627 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:17:26.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:17:26.620+0000 7f7be27a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73961) UID: 1000 2021-02-20T16:17:27.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:17:29.627 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:17:29.628 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:17:29.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:17:29.629 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:17:29.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:17:29.629 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:17:29.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:17:29.632 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T16:19:01.633 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6902965903599864) > weight (0.25) 2021-02-20T16:19:01.634 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6043470619433386) > weight (0.25) 2021-02-20T16:19:01.634 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.890383644372013) > weight (0.25) 2021-02-20T16:19:01.634 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9489584141322747) > weight (0.25) 2021-02-20T16:19:01.634 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T16:19:18.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49995191088713375) > weight (0.25) 2021-02-20T16:19:18.700 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:19:18.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:19:18.701 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:19:18.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:19:18.697+0000 7f039ecbc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74053) UID: 1000 2021-02-20T16:19:19.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:19:23.701 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:19:23.702 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:19:23.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:19:23.703 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:19:23.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:19:23.703 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:19:23.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:19:23.706 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T16:20:22.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4306671763034954) > weight (0.25) 2021-02-20T16:20:22.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7824495569885934) > weight (0.25) 2021-02-20T16:20:22.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3570151221424266) > weight (0.25) 2021-02-20T16:20:22.636 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:20:22.637 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:20:22.637 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:20:22.638 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T16:20:22.632+0000 7fd6e9873700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 73769) UID: 1000 2021-02-20T16:20:23.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:20:31.638 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:20:31.639 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:20:31.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:20:31.640 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:20:31.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:20:31.640 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:20:31.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:20:31.643 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T16:20:35.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5154523980993122) > weight (0.25) 2021-02-20T16:20:35.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7293211810480928) > weight (0.25) 2021-02-20T16:20:35.709 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:20:35.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:20:35.710 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T16:20:35.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T16:20:35.704+0000 7f6dfeae9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73084) UID: 1000 2021-02-20T16:20:36.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:20:39.711 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:20:39.712 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:20:39.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:20:39.713 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:20:39.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:20:39.713 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:20:39.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:20:39.717 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T16:21:41.717 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:21:41.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:21:41.719 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:21:41.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:21:41.714+0000 7f12f2ecd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74440) UID: 1000 2021-02-20T16:21:42.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:21:43.882 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:21:43.879+0000 7f508e15e700 -1 reset not still connected to 0x55ac91e22b10 2021-02-20T16:21:46.720 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:21:46.720 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:21:46.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:21:46.721 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:21:46.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:21:46.722 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:21:46.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:21:46.725 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T16:21:52.134 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:21:52.127+0000 7f508e15e700 -1 reset not still connected to 0x55ac7caa4680 2021-02-20T16:21:52.134 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:21:52.127+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750d190 2021-02-20T16:21:52.134 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:21:52.127+0000 7f508e15e700 -1 reset not still connected to 0x55ac918059b0 2021-02-20T16:21:53.986 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T16:21:53.982+0000 7fc273bc1700 -1 reset not still connected to 0x56117da05a00 2021-02-20T16:21:53.988 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T16:21:53.982+0000 7fc273bc1700 -1 reset not still connected to 0x56118dcbdad0 2021-02-20T16:21:53.988 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T16:21:53.982+0000 7fc273bc1700 -1 reset not still connected to 0x5611c2e082f0 2021-02-20T16:22:04.644 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5457166638870466) > weight (0.25) 2021-02-20T16:22:04.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8211554206929847) > weight (0.25) 2021-02-20T16:22:04.645 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3053601107902182) > weight (0.25) 2021-02-20T16:22:04.645 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:22:04.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:22:04.646 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:22:04.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T16:22:04.642+0000 7fa011a2c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74733) UID: 1000 2021-02-20T16:22:05.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:22:06.647 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:22:06.647 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:22:06.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:22:06.648 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:22:06.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:22:06.649 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:22:06.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:22:06.651 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T16:23:45.726 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7652757205326634) > weight (0.25) 2021-02-20T16:23:45.727 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:23:45.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:23:45.728 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:23:45.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:23:45.724+0000 7fe4946d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74633) UID: 1000 2021-02-20T16:23:46.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:23:47.729 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:23:47.729 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:23:47.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:23:47.730 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:23:47.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:23:47.731 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:23:47.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:23:47.734 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:23:56.652 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:23:56.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:23:56.653 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:23:56.654 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:23:56.647+0000 7fa43d643700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 74152) UID: 1000 2021-02-20T16:23:57.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:24:04.654 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:24:04.654 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:24:04.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:24:04.655 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:24:04.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:24:04.656 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:24:04.664 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:24:04.665 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T16:25:04.735 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.747456748698033) > weight (0.25) 2021-02-20T16:25:04.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5405569120461904) > weight (0.25) 2021-02-20T16:25:04.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6932827914019467) > weight (0.25) 2021-02-20T16:25:04.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9605852088425982) > weight (0.25) 2021-02-20T16:25:04.737 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T16:25:38.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3104736304989004) > weight (0.25) 2021-02-20T16:25:38.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9756589884536063) > weight (0.25) 2021-02-20T16:25:38.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9468355128496403) > weight (0.25) 2021-02-20T16:25:38.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.792576525875) > weight (0.25) 2021-02-20T16:25:38.667 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T16:26:43.668 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:26:43.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:26:43.670 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:26:43.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:26:43.668+0000 7f266d8a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75213) UID: 1000 2021-02-20T16:26:44.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:26:52.670 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:26:52.671 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:26:52.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:26:52.671 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:26:52.672 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:26:52.672 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:26:52.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:26:52.675 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T16:26:52.737 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3831359695722607) > weight (0.25) 2021-02-20T16:26:52.738 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:26:52.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:26:52.738 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:26:52.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:26:52.732+0000 7f9bee804700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75114) UID: 1000 2021-02-20T16:26:53.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:26:55.739 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:26:55.739 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:26:55.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:26:55.740 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:26:55.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:26:55.740 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:26:55.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:26:55.743 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T16:28:04.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8524866404397587) > weight (0.25) 2021-02-20T16:28:04.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7805629934345328) > weight (0.25) 2021-02-20T16:28:04.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.714839063400845) > weight (0.25) 2021-02-20T16:28:04.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5986470893832513) > weight (0.25) 2021-02-20T16:28:04.746 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T16:28:21.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.881078321498741) > weight (0.25) 2021-02-20T16:28:21.677 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41686777553687415) > weight (0.25) 2021-02-20T16:28:21.677 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5080549174112448) > weight (0.25) 2021-02-20T16:28:21.677 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.39145311853332443) > weight (0.25) 2021-02-20T16:28:21.677 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:29:40.747 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7143424020404842) > weight (0.25) 2021-02-20T16:29:40.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6896999852116272) > weight (0.25) 2021-02-20T16:29:40.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5914353486458498) > weight (0.25) 2021-02-20T16:29:40.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.28490146544700257) > weight (0.25) 2021-02-20T16:29:40.749 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T16:29:59.897 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:29:59.888+0000 7f803c4bb700 -1 reset not still connected to 0x55acbadbe490 2021-02-20T16:30:04.678 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:30:04.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:30:04.679 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:30:04.680 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:30:04.680 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:30:04.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:30:04.675+0000 7f06bf2ec700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75309) UID: 1000 2021-02-20T16:30:05.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:30:10.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:30:10.782 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:30:10.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:30:10.782 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:30:10.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:30:10.785 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T16:31:24.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.709242676346693) > weight (0.25) 2021-02-20T16:31:24.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9166831749259419) > weight (0.25) 2021-02-20T16:31:24.751 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T16:31:24.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T16:31:24.752 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:31:24.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T16:31:24.746+0000 7f4b0bec0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 74824) UID: 1000 2021-02-20T16:31:25.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:31:31.753 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T16:31:31.758 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:31:31.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:31:31.759 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T16:31:31.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T16:31:31.759 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T16:31:31.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T16:31:31.763 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:32:00.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3720671730420021) > weight (0.25) 2021-02-20T16:32:00.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8767163701819485) > weight (0.25) 2021-02-20T16:32:00.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.44930416067040346) > weight (0.25) 2021-02-20T16:32:00.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7784295979813813) > weight (0.25) 2021-02-20T16:32:00.787 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T16:32:36.907 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:32:36.900+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8e285b0 2021-02-20T16:32:36.908 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:32:36.900+0000 7f803c4bb700 -1 reset not still connected to 0x55acc1337670 2021-02-20T16:32:36.910 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:32:36.904+0000 7f803c4bb700 -1 reset not still connected to 0x55acd5908270 2021-02-20T16:32:36.911 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:32:36.904+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896c1a0 2021-02-20T16:33:27.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8822042108987713) > weight (0.25) 2021-02-20T16:33:27.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49668410034667876) > weight (0.25) 2021-02-20T16:33:27.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7382289666732684) > weight (0.25) 2021-02-20T16:33:27.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7748217533613128) > weight (0.25) 2021-02-20T16:33:27.771 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:33:27.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9068927830700483) > weight (0.25) 2021-02-20T16:33:27.788 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:33:27.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:33:27.790 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:33:27.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:33:27.787+0000 7f57f9784700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74540) UID: 1000 2021-02-20T16:33:28.191 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:33:34.790 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:33:34.799 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:33:34.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:33:34.800 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:33:34.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:33:34.801 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:33:34.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:33:34.808 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T16:34:37.809 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6163558344207779) > weight (0.25) 2021-02-20T16:34:37.810 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:34:37.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:34:37.811 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:34:37.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:34:37.806+0000 7f477c89b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 75693) UID: 1000 2021-02-20T16:34:38.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:34:44.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4386355063258238) > weight (0.25) 2021-02-20T16:34:44.778 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:34:44.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:34:44.779 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:34:44.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:34:44.777+0000 7fabb03a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75400) UID: 1000 2021-02-20T16:34:44.812 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:34:44.813 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:34:44.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:34:44.814 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:34:44.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:34:44.814 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:34:44.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:34:44.817 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T16:34:45.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:34:45.781 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:34:45.781 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:34:45.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:34:45.782 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:34:45.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:34:45.782 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:34:45.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:34:45.785 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T16:36:07.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7675302116183778) > weight (0.25) 2021-02-20T16:36:07.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6167765599097236) > weight (0.25) 2021-02-20T16:36:07.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4632895529968183) > weight (0.25) 2021-02-20T16:36:07.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.36449209258750515) > weight (0.25) 2021-02-20T16:36:07.829 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T16:36:38.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5703128402542595) > weight (0.25) 2021-02-20T16:36:38.787 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:36:38.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:36:38.788 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T16:36:38.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:36:38.783+0000 7f9338416700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75875) UID: 1000 2021-02-20T16:36:39.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:36:40.789 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:36:40.789 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:36:40.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:36:40.790 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:36:40.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:36:40.791 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:36:40.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:36:40.794 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:38:02.831 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4834280298169975) > weight (0.25) 2021-02-20T16:38:02.839 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:38:02.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:38:02.840 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:38:02.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:38:02.837+0000 7f14b3106700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 75784) UID: 1000 2021-02-20T16:38:03.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:38:07.841 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:38:07.842 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:38:07.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:38:07.843 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:38:07.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:38:07.844 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:38:07.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:38:07.847 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T16:38:21.795 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:38:21.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:38:21.802 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:38:21.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:38:21.797+0000 7f40cd4a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74924) UID: 1000 2021-02-20T16:38:22.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:38:29.803 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:38:29.812 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:38:29.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:38:29.812 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:38:29.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:38:29.813 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:38:29.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:38:29.816 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T16:38:30.513 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T16:38:30.497+0000 7ff4f84bf700 -1 reset not still connected to 0x55db69762be0 2021-02-20T16:38:30.513 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T16:38:30.497+0000 7ff4f84bf700 -1 reset not still connected to 0x55db69762e50 2021-02-20T16:39:31.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9721111234842571) > weight (0.25) 2021-02-20T16:39:31.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5271305662523262) > weight (0.25) 2021-02-20T16:39:31.849 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:39:31.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:39:31.850 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:39:31.850 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:39:31.850 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:39:31.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T16:39:31.843+0000 7f80e4dee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74246) UID: 1000 2021-02-20T16:39:32.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:39:37.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:39:37.961 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:39:37.962 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:39:37.962 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:39:37.965 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:39:37.965 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T16:39:38.816 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5157322689475081) > weight (0.25) 2021-02-20T16:39:38.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.54530140212719) > weight (0.25) 2021-02-20T16:39:38.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4334459343212135) > weight (0.25) 2021-02-20T16:39:38.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7045440689313195) > weight (0.25) 2021-02-20T16:39:38.818 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T16:40:47.819 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:40:47.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:40:47.831 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:40:47.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:40:47.826+0000 7fce09db0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76170) UID: 1000 2021-02-20T16:40:48.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:40:48.832 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:40:48.833 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:40:48.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:40:48.834 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:40:48.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:40:48.834 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:40:48.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:40:48.837 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T16:41:20.967 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3812300315891063) > weight (0.25) 2021-02-20T16:41:20.975 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:41:20.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:41:20.976 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:41:20.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:41:20.973+0000 7f6d86e8a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76079) UID: 1000 2021-02-20T16:41:21.377 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:41:25.977 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:41:25.982 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:41:25.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:41:25.983 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:41:25.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:41:25.984 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:41:25.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:41:25.987 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T16:42:08.838 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:42:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:42:08.849 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:42:08.849 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:42:08.849 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:42:08.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:42:08.844+0000 7f350b087700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76361) UID: 1000 2021-02-20T16:42:09.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:42:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:42:14.954 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:42:14.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:42:14.955 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:42:14.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:42:14.958 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T16:42:20.676 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:42:20.670+0000 7f508e15e700 -1 reset not still connected to 0x55ac7caa4680 2021-02-20T16:42:20.676 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:42:20.670+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750d190 2021-02-20T16:42:41.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4667640383423953) > weight (0.25) 2021-02-20T16:42:41.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6217892610929909) > weight (0.25) 2021-02-20T16:42:41.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6924502974252676) > weight (0.25) 2021-02-20T16:42:41.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.33598367383487004) > weight (0.25) 2021-02-20T16:42:41.999 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T16:43:52.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3000859999560225) > weight (0.25) 2021-02-20T16:43:52.960 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:43:52.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:43:52.962 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:43:52.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:43:52.958+0000 7faa56dbb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75978) UID: 1000 2021-02-20T16:43:53.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:44:00.963 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:44:00.963 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:44:00.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:44:00.964 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:44:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:44:00.965 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:44:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:44:00.968 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T16:44:22.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42244807752415703) > weight (0.25) 2021-02-20T16:44:22.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9564095786788475) > weight (0.25) 2021-02-20T16:44:22.002 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:44:22.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:44:22.003 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:44:22.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T16:44:21.997+0000 7f4b3c480700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 76269) UID: 1000 2021-02-20T16:44:22.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:44:23.004 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:44:23.004 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:44:23.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:44:23.005 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:44:23.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:44:23.006 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:44:23.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:44:23.008 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T16:45:56.969 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9573184925386102) > weight (0.25) 2021-02-20T16:45:56.970 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:45:56.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:45:56.971 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T16:45:56.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:45:56.963+0000 7fe4bca54700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76719) UID: 1000 2021-02-20T16:45:57.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:45:57.971 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:45:57.972 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:45:57.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:45:57.973 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:45:57.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:45:57.973 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:45:57.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:45:57.977 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T16:46:09.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.882986213279988) > weight (0.25) 2021-02-20T16:46:09.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3068227859411038) > weight (0.25) 2021-02-20T16:46:09.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3106045422476261) > weight (0.25) 2021-02-20T16:46:09.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8565039142330152) > weight (0.25) 2021-02-20T16:46:09.010 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T16:47:26.978 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6785014653689728) > weight (0.25) 2021-02-20T16:47:26.979 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:47:26.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:47:26.981 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:47:26.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:47:26.977+0000 7f0967942700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76912) UID: 1000 2021-02-20T16:47:27.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:47:34.982 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:47:34.982 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:47:34.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:47:34.983 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:47:34.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:47:34.984 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:47:34.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:47:34.987 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T16:47:43.011 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:47:43.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:47:43.013 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T16:47:43.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:47:43.009+0000 7fbe6c7ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75501) UID: 1000 2021-02-20T16:47:43.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:47:49.014 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:47:49.015 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:47:49.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:47:49.015 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:47:49.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:47:49.016 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:47:49.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:47:49.019 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T16:49:03.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.707915522750546) > weight (0.25) 2021-02-20T16:49:03.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.34703518871486594) > weight (0.25) 2021-02-20T16:49:03.021 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:49:03.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:49:03.022 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:49:03.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T16:49:03.019+0000 7f2577db0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 76818) UID: 1000 2021-02-20T16:49:03.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:49:11.024 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:49:11.024 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:49:11.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:49:11.025 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:49:11.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:49:11.026 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:49:11.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:49:11.028 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T16:49:12.988 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6698976811792856) > weight (0.25) 2021-02-20T16:49:12.988 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:49:12.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:49:12.989 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:49:12.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:49:12.987+0000 7f8fe7e5f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77012) UID: 1000 2021-02-20T16:49:13.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:49:17.990 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:49:17.990 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:49:17.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:49:17.991 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:49:17.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:49:17.992 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:49:17.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:49:17.994 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T16:50:13.825 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:13.819+0000 7f508e15e700 -1 reset not still connected to 0x55ac7caa4680 2021-02-20T16:50:13.826 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:13.819+0000 7f508e15e700 -1 reset not still connected to 0x55ac8750d190 2021-02-20T16:50:13.826 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:13.819+0000 7f508e15e700 -1 reset not still connected to 0x55ac8a6b43c0 2021-02-20T16:50:13.827 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:13.823+0000 7f508e15e700 -1 reset not still connected to 0x55ac8a6b5330 2021-02-20T16:50:13.828 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:13.823+0000 7f508e15e700 -1 reset not still connected to 0x55ac8bccb1e0 2021-02-20T16:50:52.030 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T16:50:52.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T16:50:52.032 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T16:50:52.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T16:50:52.028+0000 7f2e478c7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77111) UID: 1000 2021-02-20T16:50:52.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:50:52.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9346050612001814) > weight (0.25) 2021-02-20T16:50:52.996 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T16:50:52.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T16:50:52.996 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:50:52.997 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T16:50:52.997 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:50:52.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T16:50:52.992+0000 7f4262266700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77295) UID: 1000 2021-02-20T16:50:53.075 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:53.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac8e305ba0 2021-02-20T16:50:53.075 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T16:50:53.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac8e305d40 2021-02-20T16:50:53.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:50:57.033 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T16:50:57.033 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:50:57.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:50:57.034 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T16:50:57.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T16:50:57.035 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T16:50:57.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T16:50:57.038 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T16:50:59.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:50:59.099 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T16:50:59.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T16:50:59.099 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T16:50:59.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T16:50:59.102 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T16:52:47.103 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:52:47.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:52:47.105 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T16:52:47.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:52:47.102+0000 7f7952026700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 76552) UID: 1000 2021-02-20T16:52:47.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:52:48.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.49707820427307436) > weight (0.25) 2021-02-20T16:52:48.039 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T16:52:48.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T16:52:48.040 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T16:52:48.041 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T16:52:48.038+0000 7f1db2dfb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76461) UID: 1000 2021-02-20T16:52:48.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:52:51.041 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T16:52:51.041 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:52:51.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:52:51.042 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T16:52:51.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T16:52:51.043 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T16:52:51.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T16:52:51.046 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T16:52:54.106 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:52:54.106 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:52:54.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:52:54.107 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:52:54.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:52:54.107 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:52:54.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:52:54.110 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T16:54:01.112 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:54:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:54:01.114 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:54:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:54:01.108+0000 7f821e51c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77676) UID: 1000 2021-02-20T16:54:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:54:10.115 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:54:10.116 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:54:10.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:54:10.116 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:54:10.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:54:10.117 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:54:10.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:54:10.120 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T16:54:28.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.469431002512269) > weight (0.25) 2021-02-20T16:54:28.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6655429690576135) > weight (0.25) 2021-02-20T16:54:28.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9404035391626833) > weight (0.25) 2021-02-20T16:54:28.048 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T16:54:28.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T16:54:28.049 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T16:54:28.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T16:54:28.048+0000 7f3f39c6d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 75023) UID: 1000 2021-02-20T16:54:28.450 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:54:36.050 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T16:54:36.050 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:54:36.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:54:36.051 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T16:54:36.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T16:54:36.051 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T16:54:36.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T16:54:36.055 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T16:55:44.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.526474687422257) > weight (0.25) 2021-02-20T16:55:44.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9501537774309449) > weight (0.25) 2021-02-20T16:55:44.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26650187835808903) > weight (0.25) 2021-02-20T16:55:44.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.288761543639817) > weight (0.25) 2021-02-20T16:55:44.057 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T16:55:57.121 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T16:55:57.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T16:55:57.123 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T16:55:57.123 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T16:55:57.123 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:55:57.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T16:55:57.122+0000 7f359d49a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77776) UID: 1000 2021-02-20T16:55:57.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:56:03.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:56:03.225 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T16:56:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T16:56:03.226 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T16:56:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T16:56:03.229 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T16:56:57.308 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:56:57.297+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b2680 2021-02-20T16:56:57.309 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:56:57.297+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b2a90 2021-02-20T16:56:57.309 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:56:57.297+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b3380 2021-02-20T16:56:57.309 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T16:56:57.297+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b5a80 2021-02-20T16:57:01.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9706860098235887) > weight (0.25) 2021-02-20T16:57:01.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3805461701848153) > weight (0.25) 2021-02-20T16:57:01.059 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T16:57:01.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T16:57:01.060 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T16:57:01.061 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T16:57:01.057+0000 7fd320ada700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 77204) UID: 1000 2021-02-20T16:57:01.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T16:57:10.061 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T16:57:10.062 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T16:57:10.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T16:57:10.063 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T16:57:10.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T16:57:10.064 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T16:57:10.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T16:57:10.067 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T16:57:22.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6835107720536326) > weight (0.25) 2021-02-20T16:57:22.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8768663676270066) > weight (0.25) 2021-02-20T16:57:22.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4465056036049705) > weight (0.25) 2021-02-20T16:57:22.231 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.35509739524674233) > weight (0.25) 2021-02-20T16:57:22.231 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T16:58:35.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37398382337590863) > weight (0.25) 2021-02-20T16:58:35.069 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5534578990615939) > weight (0.25) 2021-02-20T16:58:35.069 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.86298118845123) > weight (0.25) 2021-02-20T16:58:35.070 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9563059735608983) > weight (0.25) 2021-02-20T16:58:35.070 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T16:58:54.232 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9046416702508376) > weight (0.25) 2021-02-20T16:58:54.233 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.616297259983954) > weight (0.25) 2021-02-20T16:58:54.233 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.47372459996929805) > weight (0.25) 2021-02-20T16:58:54.233 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5582979492724935) > weight (0.25) 2021-02-20T16:58:54.233 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T17:00:03.071 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5926684751715703) > weight (0.25) 2021-02-20T17:00:03.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6948279065039241) > weight (0.25) 2021-02-20T17:00:03.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5720733338901985) > weight (0.25) 2021-02-20T17:00:03.073 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:00:03.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:00:03.074 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:00:03.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:00:03.069+0000 7f5f3cf2e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 77875) UID: 1000 2021-02-20T17:00:03.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:00:09.074 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:00:09.075 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:00:09.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:00:09.076 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:00:09.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:00:09.077 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:00:09.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:00:09.080 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T17:00:15.235 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:00:15.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:00:15.236 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:00:15.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:00:15.232+0000 7fecc6714700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77970) UID: 1000 2021-02-20T17:00:15.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:00:16.237 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:00:16.238 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:00:16.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:00:16.238 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:00:16.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:00:16.239 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:00:16.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:00:16.242 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:01:26.081 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:01:26.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:01:26.083 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:01:26.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:01:26.083+0000 7f68412a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77394) UID: 1000 2021-02-20T17:01:26.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:01:29.085 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:01:29.085 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:01:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:01:29.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:01:29.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:01:29.087 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:01:29.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:01:29.090 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T17:02:00.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9384438176837475) > weight (0.25) 2021-02-20T17:02:00.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42110287580250116) > weight (0.25) 2021-02-20T17:02:00.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.55967649500597) > weight (0.25) 2021-02-20T17:02:00.244 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:02:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:02:00.245 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:02:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:02:00.242+0000 7f69ccb1a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 73183) UID: 1000 2021-02-20T17:02:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:02:05.246 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:02:05.247 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:02:05.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:02:05.248 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:02:05.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:02:05.248 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:02:05.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:02:05.251 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:02:33.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:02:33.661+0000 7fd70bf23700 -1 rbd::mirror::ImageReplayer: 0x55f462175c00 [3/e450489d-07e5-4769-bd71-337efe806c96] start: already running 2021-02-20T17:03:04.091 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:03:04.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:03:04.093 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:03:04.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:03:04.089+0000 7fc50bd43700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78354) UID: 1000 2021-02-20T17:03:04.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:03:13.094 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:03:13.094 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:03:13.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:03:13.095 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:03:13.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:03:13.096 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:03:13.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:03:13.099 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T17:03:16.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4348005735985746) > weight (0.25) 2021-02-20T17:03:16.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.27224437436905224) > weight (0.25) 2021-02-20T17:03:16.253 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:03:16.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:03:16.254 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:03:16.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:03:16.248+0000 7fd70cf25700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75593) UID: 1000 2021-02-20T17:03:16.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:03:21.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:03:21.255 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:03:21.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:03:21.256 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:03:21.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:03:21.257 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:03:21.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:03:21.261 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:04:16.100 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:04:16.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:04:16.103 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:04:16.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:04:16.095+0000 7f1993746700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78545) UID: 1000 2021-02-20T17:04:16.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:04:19.104 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:04:19.104 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:04:19.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:04:19.105 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:04:19.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:04:19.106 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:04:19.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:04:19.109 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T17:04:30.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9796019410534282) > weight (0.25) 2021-02-20T17:04:30.263 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:04:30.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:04:30.264 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:04:30.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:04:30.263+0000 7fe27e63b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77485) UID: 1000 2021-02-20T17:04:30.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:04:31.265 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:04:31.265 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:04:31.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:04:31.266 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:04:31.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:04:31.267 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:04:31.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:04:31.270 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T17:04:44.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:04:44.662+0000 7f30db372700 -1 rbd::mirror::ImageReplayer: 0x55900a5fcf80 [3/eb5ee405-f45d-4109-b887-4acc3932b673] start: already running 2021-02-20T17:05:38.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5511928239620956) > weight (0.25) 2021-02-20T17:05:38.272 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:05:38.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:05:38.273 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:05:38.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:05:38.269+0000 7f3db5135700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 78826) UID: 1000 2021-02-20T17:05:38.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:05:44.274 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:05:44.274 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:05:44.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:05:44.275 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:05:44.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:05:44.276 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:05:44.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:05:44.278 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T17:05:57.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.874616091002753) > weight (0.25) 2021-02-20T17:05:57.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6647282456410619) > weight (0.25) 2021-02-20T17:05:57.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.35403845789395183) > weight (0.25) 2021-02-20T17:05:57.111 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:05:57.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:05:57.112 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:05:57.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:05:57.109+0000 7f2dd62ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 78164) UID: 1000 2021-02-20T17:05:57.514 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:05:59.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:05:59.114 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:05:59.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:05:59.115 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:05:59.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:05:59.115 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:05:59.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:05:59.118 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T17:07:13.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6702782531833014) > weight (0.25) 2021-02-20T17:07:13.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5096936945081026) > weight (0.25) 2021-02-20T17:07:13.122 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:07:13.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:07:13.123 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:07:13.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:07:13.119+0000 7f30dc374700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 78069) UID: 1000 2021-02-20T17:07:13.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:07:15.124 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:07:15.125 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:07:15.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:07:15.126 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:07:15.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:07:15.126 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:07:15.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:07:15.129 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T17:07:16.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4546703233464182) > weight (0.25) 2021-02-20T17:07:16.280 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:07:16.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:07:16.281 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:07:16.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:07:16.279+0000 7fb983736700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 78929) UID: 1000 2021-02-20T17:07:16.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:07:18.281 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:07:18.282 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:07:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:07:18.283 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:07:18.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:07:18.283 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:07:18.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:07:18.286 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T17:08:28.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2911812355451129) > weight (0.25) 2021-02-20T17:08:28.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.47779910280288984) > weight (0.25) 2021-02-20T17:08:28.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39167602168217) > weight (0.25) 2021-02-20T17:08:28.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9944471643646737) > weight (0.25) 2021-02-20T17:08:28.132 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:08:53.287 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:08:53.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:08:53.288 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:08:53.288 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:08:53.289 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:08:53.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:08:53.285+0000 7fa678d81700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78255) UID: 1000 2021-02-20T17:08:53.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:08:59.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:08:59.391 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:08:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:08:59.392 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:08:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:08:59.394 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:09:30.132 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5653997560511782) > weight (0.25) 2021-02-20T17:09:30.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7408996304521249) > weight (0.25) 2021-02-20T17:09:30.133 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8239603000057326) > weight (0.25) 2021-02-20T17:09:30.133 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:09:30.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:09:30.134 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:09:30.134 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:09:30.135 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:09:30.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:09:30.128+0000 7ff797cb0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 79028) UID: 1000 2021-02-20T17:09:30.536 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:09:36.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:09:36.236 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:09:36.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:09:36.237 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:09:36.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:09:36.240 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T17:10:43.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43563400533601604) > weight (0.25) 2021-02-20T17:10:43.396 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:10:43.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:10:43.397 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:10:43.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:10:43.395+0000 7feb65190700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79211) UID: 1000 2021-02-20T17:10:43.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:10:51.398 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:10:51.398 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:10:51.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:10:51.399 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:10:51.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:10:51.400 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:10:51.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:10:51.403 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T17:11:00.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3598338812998225) > weight (0.25) 2021-02-20T17:11:00.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.71466580648523) > weight (0.25) 2021-02-20T17:11:00.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6832620298267086) > weight (0.25) 2021-02-20T17:11:00.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5490438890869727) > weight (0.25) 2021-02-20T17:11:00.242 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:12:11.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9541056496318394) > weight (0.25) 2021-02-20T17:12:11.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8847372580531024) > weight (0.25) 2021-02-20T17:12:11.244 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:12:11.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:12:11.245 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:12:11.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:12:11.241+0000 7fd23d369700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79120) UID: 1000 2021-02-20T17:12:11.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:12:18.245 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:12:18.246 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:12:18.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:12:18.247 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:12:18.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:12:18.247 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:12:18.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:12:18.251 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T17:12:35.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9956216801643908) > weight (0.25) 2021-02-20T17:12:35.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4366465959003867) > weight (0.25) 2021-02-20T17:12:35.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9352526079089963) > weight (0.25) 2021-02-20T17:12:35.406 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.44030187299117185) > weight (0.25) 2021-02-20T17:12:35.406 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T17:13:47.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.707971958890738) > weight (0.25) 2021-02-20T17:13:47.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.31954592348476074) > weight (0.25) 2021-02-20T17:13:47.253 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:13:47.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:13:47.255 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:13:47.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:13:47.255+0000 7f31005ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79602) UID: 1000 2021-02-20T17:13:47.657 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:13:50.256 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:13:50.256 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:13:50.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:13:50.257 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:13:50.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:13:50.258 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:13:50.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:13:50.261 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:14:26.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48696981193736133) > weight (0.25) 2021-02-20T17:14:26.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5385393956114319) > weight (0.25) 2021-02-20T17:14:26.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4750681426548299) > weight (0.25) 2021-02-20T17:14:26.408 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6695980081593016) > weight (0.25) 2021-02-20T17:14:26.408 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T17:15:20.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3468529813693282) > weight (0.25) 2021-02-20T17:15:20.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9424221417909455) > weight (0.25) 2021-02-20T17:15:20.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8034892295797481) > weight (0.25) 2021-02-20T17:15:20.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7385995047267483) > weight (0.25) 2021-02-20T17:15:20.264 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T17:15:29.409 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7897874807688725) > weight (0.25) 2021-02-20T17:15:29.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2639695592276562) > weight (0.25) 2021-02-20T17:15:29.410 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:15:29.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:15:29.412 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:15:29.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:15:29.409+0000 7fd095203700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 78636) UID: 1000 2021-02-20T17:15:29.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:15:36.413 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:15:36.413 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:15:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:15:36.414 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:15:36.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:15:36.415 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:15:36.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:15:36.419 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:16:33.265 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7954862279280949) > weight (0.25) 2021-02-20T17:16:33.266 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25034589838853527) > weight (0.25) 2021-02-20T17:16:33.266 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.897056926059086) > weight (0.25) 2021-02-20T17:16:33.267 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6308979846101692) > weight (0.25) 2021-02-20T17:16:33.267 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T17:17:06.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4006893542745785) > weight (0.25) 2021-02-20T17:17:06.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7861558681120091) > weight (0.25) 2021-02-20T17:17:06.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4129956139817631) > weight (0.25) 2021-02-20T17:17:06.421 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:17:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:17:06.422 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:17:06.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:17:06.418+0000 7f282143e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 78445) UID: 1000 2021-02-20T17:17:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:17:11.423 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:17:11.423 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:17:11.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:17:11.424 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:17:11.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:17:11.425 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:17:11.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:17:11.428 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T17:18:32.268 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:18:32.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:18:32.270 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:18:32.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:18:32.265+0000 7f3d9481f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78735) UID: 1000 2021-02-20T17:18:32.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:18:39.271 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:18:39.272 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:18:39.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:18:39.273 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:18:39.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:18:39.273 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:18:39.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:18:39.277 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T17:18:58.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7626878740810082) > weight (0.25) 2021-02-20T17:18:58.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6803320060304308) > weight (0.25) 2021-02-20T17:18:58.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5732438616220947) > weight (0.25) 2021-02-20T17:18:58.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6460468185027054) > weight (0.25) 2021-02-20T17:18:58.430 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:19:46.753 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:19:46.749+0000 7f508e15e700 -1 reset not still connected to 0x55ac7e7fc680 2021-02-20T17:19:46.758 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:19:46.753+0000 7f508e15e700 -1 reset not still connected to 0x55ac82dfd110 2021-02-20T17:19:46.758 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:19:46.753+0000 7f508e15e700 -1 reset not still connected to 0x55ac8e3075a0 2021-02-20T17:20:21.885 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:21.880+0000 7f508e15e700 -1 reset not still connected to 0x55ac8215f670 2021-02-20T17:20:21.886 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:21.880+0000 7f508e15e700 -1 reset not still connected to 0x55ac82dfd110 2021-02-20T17:20:21.887 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:21.880+0000 7f508e15e700 -1 reset not still connected to 0x55ac88d97790 2021-02-20T17:20:21.888 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:21.880+0000 7f508e15e700 -1 reset not still connected to 0x55ac9168ec30 2021-02-20T17:20:28.431 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:20:28.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:20:28.433 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:20:28.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:20:28.430+0000 7f6c6a4f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79311) UID: 1000 2021-02-20T17:20:28.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:30.434 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:20:30.434 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:20:30.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:30.435 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:20:30.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:20:30.436 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:20:30.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:20:30.439 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T17:20:33.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9034165281703473) > weight (0.25) 2021-02-20T17:20:33.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3753083635617802) > weight (0.25) 2021-02-20T17:20:33.280 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:20:33.281 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:20:33.281 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:20:33.291 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:20:33.286+0000 7f6d2c9d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79694) UID: 1000 2021-02-20T17:20:33.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:20:36.283 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:20:36.283 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:20:36.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:20:36.284 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:20:36.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:20:36.285 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:20:36.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:20:36.288 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T17:20:36.645 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:36.640+0000 7f508e15e700 -1 reset not still connected to 0x55ac87199450 2021-02-20T17:20:36.648 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:36.640+0000 7f508e15e700 -1 reset not still connected to 0x55ac881641a0 2021-02-20T17:20:36.648 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:20:36.640+0000 7f508e15e700 -1 reset not still connected to 0x55ac8d16d330 2021-02-20T17:20:59.114 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T17:20:59.103+0000 7f1c785e8700 -1 reset not still connected to 0x555a6fdad2b0 2021-02-20T17:20:59.115 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T17:20:59.103+0000 7f1c785e8700 -1 reset not still connected to 0x555a6fdada00 2021-02-20T17:20:59.115 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T17:20:59.103+0000 7f1c785e8700 -1 reset not still connected to 0x555a6fdaf260 2021-02-20T17:20:59.115 INFO:tasks.ceph.cluster1.osd.0.smithi106.stderr:2021-02-20T17:20:59.103+0000 7f1c785e8700 -1 reset not still connected to 0x555a701ce630 2021-02-20T17:21:09.540 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:21:09.529+0000 7f803c4bb700 -1 reset not still connected to 0x55acb88ecc30 2021-02-20T17:21:47.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3647761510315718) > weight (0.25) 2021-02-20T17:21:47.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5000180101606411) > weight (0.25) 2021-02-20T17:21:47.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4360430432094712) > weight (0.25) 2021-02-20T17:21:47.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.909751616862205) > weight (0.25) 2021-02-20T17:21:47.290 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T17:22:00.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3438221971841896) > weight (0.25) 2021-02-20T17:22:00.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8181306821384146) > weight (0.25) 2021-02-20T17:22:00.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7668413465819085) > weight (0.25) 2021-02-20T17:22:00.869 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:22:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:22:00.870 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:22:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:22:00.864+0000 7fb5a79d3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 79892) UID: 1000 2021-02-20T17:22:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:22:09.871 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:22:09.872 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:22:09.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:22:09.873 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:22:09.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:22:09.874 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:22:09.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:22:09.877 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T17:23:34.878 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:23:34.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:23:34.880 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:23:34.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:23:34.878+0000 7f39730e8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80085) UID: 1000 2021-02-20T17:23:35.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:23:36.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43429893500426686) > weight (0.25) 2021-02-20T17:23:36.292 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.729232465558286) > weight (0.25) 2021-02-20T17:23:36.292 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T17:23:36.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T17:23:36.293 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:23:36.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:23:36.290+0000 7f3e2bba6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80184) UID: 1000 2021-02-20T17:23:36.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:23:39.294 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T17:23:39.295 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:23:39.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:23:39.295 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T17:23:39.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T17:23:39.296 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T17:23:39.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T17:23:39.304 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T17:23:41.881 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:23:41.881 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:23:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:23:41.882 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:23:41.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:23:41.883 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:23:41.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:23:41.886 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T17:25:24.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41475928950473206) > weight (0.25) 2021-02-20T17:25:24.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4746837939916949) > weight (0.25) 2021-02-20T17:25:24.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.486548667272796) > weight (0.25) 2021-02-20T17:25:24.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5354046043047799) > weight (0.25) 2021-02-20T17:25:24.307 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T17:25:30.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3456930766937627) > weight (0.25) 2021-02-20T17:25:30.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6923496983477219) > weight (0.25) 2021-02-20T17:25:30.887 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6940487699356563) > weight (0.25) 2021-02-20T17:25:30.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40030391189960324) > weight (0.25) 2021-02-20T17:25:30.888 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T17:26:28.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4925630518687225) > weight (0.25) 2021-02-20T17:26:28.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3725104024551742) > weight (0.25) 2021-02-20T17:26:28.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.394497375510679) > weight (0.25) 2021-02-20T17:26:28.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6729314650954935) > weight (0.25) 2021-02-20T17:26:28.309 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T17:27:20.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8262194502117511) > weight (0.25) 2021-02-20T17:27:20.890 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:27:20.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:27:20.891 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:27:20.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:27:20.889+0000 7ff1107e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79503) UID: 1000 2021-02-20T17:27:21.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:27:21.320 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:27:21.314+0000 7f508e15e700 -1 reset not still connected to 0x55ac88164270 2021-02-20T17:27:22.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:27:22.161+0000 7f561ed8a700 -1 rbd::mirror::ImageReplayer: 0x55d279d02f00 [3/3ca6eed3-b4cc-42a3-8669-fd637c7a8d69] start: already running 2021-02-20T17:27:22.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:27:22.161+0000 7f561f58b700 -1 rbd::mirror::ImageReplayer: 0x55d279d03b80 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T17:27:22.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:27:22.161+0000 7f561ed8a700 -1 rbd::mirror::ImageReplayer: 0x55d279d03180 [3/4d947326-1304-4f75-9cb5-1cc9895ee2c8] start: already running 2021-02-20T17:27:22.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:27:22.165+0000 7f561f58b700 -1 rbd::mirror::ImageReplayer: 0x55d279d04080 [3/5d895514-421a-41d1-8f87-96dea12501c8] start: already running 2021-02-20T17:27:22.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:27:22.165+0000 7f561f58b700 -1 rbd::mirror::ImageReplayer: 0x55d279d03400 [3/6a6f5866-70c0-4063-8b45-7c49d8fcdf4b] start: already running 2021-02-20T17:27:22.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.185+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f81dedd680 [3/3610e9ef-89dd-42ef-8d69-0d79bfec3e56] start: already running 2021-02-20T17:27:22.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f81dedd400 [3/4c4f4929-06c7-4203-81c6-a0c9560ab291] start: already running 2021-02-20T17:27:22.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f81deded00 [3/75792560-55ec-454a-bb3c-f7a23e6221d8] start: already running 2021-02-20T17:27:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f822198000 [3/828e1151-fa83-405f-a6cf-ec7c06ae0dee] start: already running 2021-02-20T17:27:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f81dedd900 [3/a2357b67-5e9d-4bab-b496-4a5956203c0c] start: already running 2021-02-20T17:27:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f81dedde00 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T17:27:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f821fffb80 [3/b5d77140-8b49-4567-b4bc-f2d77cea0806] start: already running 2021-02-20T17:27:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f821fffe00 [3/d9d942d6-60aa-4cd8-a77e-61392adbcc0a] start: already running 2021-02-20T17:27:22.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f821ffef00 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T17:27:22.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f821ffe780 [3/e450489d-07e5-4769-bd71-337efe806c96] start: already running 2021-02-20T17:27:22.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f64fc675700 -1 rbd::mirror::ImageReplayer: 0x55f822198280 [3/fc56e3ca-8114-4f4e-9bba-183b58b29c89] start: already running 2021-02-20T17:27:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8830f00 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T17:27:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831180 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T17:27:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831e00 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T17:27:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595ec8c0c80 [3/3738c109-5122-450d-aad4-0ba00b069fb4] start: already running 2021-02-20T17:27:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831400 [3/3a7a8fa6-6a87-4c14-b1a5-91640b3311d6] start: already running 2021-02-20T17:27:22.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8830c80 [3/081dcf2b-6aec-4ae9-b0f2-90a8a2eab556] start: already running 2021-02-20T17:27:22.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8830f00 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T17:27:22.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831180 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T17:27:22.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831e00 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T17:27:22.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595ec8c0c80 [3/3738c109-5122-450d-aad4-0ba00b069fb4] start: already running 2021-02-20T17:27:22.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.205+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831400 [3/3a7a8fa6-6a87-4c14-b1a5-91640b3311d6] start: already running 2021-02-20T17:27:22.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.221+0000 7f96e9ae1700 -1 rbd::mirror::ImageReplayer: 0x5595e8832080 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T17:27:22.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.221+0000 7f96e9ae1700 -1 rbd::mirror::ImageReplayer: 0x5595e8831680 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T17:27:22.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.221+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831900 [3/90d8a604-9cc2-4227-8cf3-7e7ff6b73bd0] start: already running 2021-02-20T17:27:22.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:27:22.221+0000 7f96e92e0700 -1 rbd::mirror::ImageReplayer: 0x5595e8831b80 [3/91cf5a1a-070a-45f0-9e15-e0e31870c4e1] start: already running 2021-02-20T17:27:28.893 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:27:28.893 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:27:28.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:27:28.894 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:27:28.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:27:28.895 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:27:28.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:27:28.898 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T17:27:59.310 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:27:59.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:27:59.312 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:27:59.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:27:59.308+0000 7f776666a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79992) UID: 1000 2021-02-20T17:27:59.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:28:06.313 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:28:06.314 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:28:06.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:28:06.314 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:28:06.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:28:06.315 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:28:06.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:28:06.317 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T17:29:14.318 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:29:14.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:29:14.325 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:29:14.325 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:29:14.325 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:29:14.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:29:14.323+0000 7f042e59e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80670) UID: 1000 2021-02-20T17:29:14.726 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:29:18.899 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:29:18.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:29:18.900 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:29:18.900 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:29:18.900 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:29:18.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:29:18.899+0000 7f561fd8c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80466) UID: 1000 2021-02-20T17:29:19.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:29:20.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:29:20.427 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:29:20.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:29:20.428 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:29:20.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:29:20.431 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T17:29:25.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:29:25.003 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:29:25.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:29:25.004 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:29:25.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:29:25.007 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-02-20T17:30:06.510 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:30:06.498+0000 7f803c4bb700 -1 reset not still connected to 0x55acb9174750 2021-02-20T17:30:06.510 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:30:06.498+0000 7f803c4bb700 -1 reset not still connected to 0x55acbadbcf70 2021-02-20T17:30:06.510 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:30:06.498+0000 7f803c4bb700 -1 reset not still connected to 0x55acc9694270 2021-02-20T17:30:06.511 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:30:06.498+0000 7f803c4bb700 -1 reset not still connected to 0x55accab01ba0 2021-02-20T17:30:06.511 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:30:06.498+0000 7f803c4bb700 -1 reset not still connected to 0x55accab01e10 2021-02-20T17:30:59.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8848788427700258) > weight (0.25) 2021-02-20T17:30:59.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.645773328843208) > weight (0.25) 2021-02-20T17:30:59.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32289961312469373) > weight (0.25) 2021-02-20T17:30:59.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4755956684122562) > weight (0.25) 2021-02-20T17:30:59.017 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:31:15.432 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4623792281214384) > weight (0.25) 2021-02-20T17:31:15.440 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:31:15.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:31:15.442 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:31:15.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T17:31:15.436+0000 7faf46d47700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 77585) UID: 1000 2021-02-20T17:31:15.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:31:20.443 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:31:20.443 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:31:20.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:31:20.444 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:31:20.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:31:20.445 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:31:20.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:31:20.448 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T17:32:08.017 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:32:08.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:32:08.047 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:32:08.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:32:08.027+0000 7f7b9ebe0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80852) UID: 1000 2021-02-20T17:32:08.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:32:09.581 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:32:09.575+0000 7f803c4bb700 -1 reset not still connected to 0x55acc13351e0 2021-02-20T17:32:09.581 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:32:09.575+0000 7f803c4bb700 -1 reset not still connected to 0x55acc13378e0 2021-02-20T17:32:09.582 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:32:09.575+0000 7f803c4bb700 -1 reset not still connected to 0x55accab01930 2021-02-20T17:32:15.049 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:32:15.049 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:32:15.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:32:15.051 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:32:15.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:32:15.052 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:32:15.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:32:15.056 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T17:32:55.449 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6775460081204585) > weight (0.25) 2021-02-20T17:32:55.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32873423601245555) > weight (0.25) 2021-02-20T17:32:55.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8279911290965233) > weight (0.25) 2021-02-20T17:32:55.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.547557032491217) > weight (0.25) 2021-02-20T17:32:55.459 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T17:33:39.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30787700857443556) > weight (0.25) 2021-02-20T17:33:39.070 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42396818414417925) > weight (0.25) 2021-02-20T17:33:39.070 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46094502158124406) > weight (0.25) 2021-02-20T17:33:39.071 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:33:39.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:33:39.073 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:33:39.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:33:39.069+0000 7f96ea2e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 80276) UID: 1000 2021-02-20T17:33:39.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:33:43.075 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:33:43.080 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:33:43.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:33:43.081 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:33:43.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:33:43.082 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:33:43.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:33:43.085 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:33:55.460 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:33:55.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:33:55.470 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:33:55.471 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:33:55.465+0000 7f3aaff79700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80761) UID: 1000 2021-02-20T17:33:55.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:34:03.470 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:34:03.471 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:34:03.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:34:03.472 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:34:03.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:34:03.472 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:34:03.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:34:03.475 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T17:34:53.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8871910204695407) > weight (0.25) 2021-02-20T17:34:53.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.27740657167373517) > weight (0.25) 2021-02-20T17:34:53.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5124489755561363) > weight (0.25) 2021-02-20T17:34:53.087 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:34:53.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:34:53.088 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T17:34:53.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:34:53.083+0000 7f652690a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81144) UID: 1000 2021-02-20T17:34:53.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:34:55.089 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:34:55.090 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:34:55.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:34:55.091 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:34:55.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:34:55.091 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:34:55.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:34:55.094 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-02-20T17:35:02.666 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:35:02.659+0000 7f508e15e700 -1 reset not still connected to 0x55ac91404560 2021-02-20T17:35:46.476 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8551018603208157) > weight (0.25) 2021-02-20T17:35:46.477 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8104754889890599) > weight (0.25) 2021-02-20T17:35:46.477 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6616077159478028) > weight (0.25) 2021-02-20T17:35:46.477 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:35:46.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:35:46.478 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:35:46.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:35:46.474+0000 7fca01a0e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 79410) UID: 1000 2021-02-20T17:35:46.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:35:52.480 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:35:52.480 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:35:52.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:35:52.481 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:35:52.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:35:52.482 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:35:52.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:35:52.485 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T17:36:54.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7503003996804586) > weight (0.25) 2021-02-20T17:36:54.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5727887001468543) > weight (0.25) 2021-02-20T17:36:54.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5101358738634251) > weight (0.25) 2021-02-20T17:36:54.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.28587602294232206) > weight (0.25) 2021-02-20T17:36:54.101 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-02-20T17:37:19.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7386166897041994) > weight (0.25) 2021-02-20T17:37:19.486 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:37:19.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:37:19.487 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:37:19.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T17:37:19.488+0000 7f995ab3a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 80953) UID: 1000 2021-02-20T17:37:19.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:37:22.488 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:37:22.488 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:37:22.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:37:22.489 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:37:22.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:37:22.490 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:37:22.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:37:22.493 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:38:24.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9454624564689454) > weight (0.25) 2021-02-20T17:38:24.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8746164512432805) > weight (0.25) 2021-02-20T17:38:24.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6705120773446639) > weight (0.25) 2021-02-20T17:38:24.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.932737742063286) > weight (0.25) 2021-02-20T17:38:24.506 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-02-20T17:38:35.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4718572383707531) > weight (0.25) 2021-02-20T17:38:35.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9907585486442315) > weight (0.25) 2021-02-20T17:38:35.108 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:38:35.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:38:35.109 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:38:35.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:38:35.107+0000 7f64fd677700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 79789) UID: 1000 2021-02-20T17:38:35.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:38:38.111 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:38:38.111 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:38:38.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:38:38.112 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:38:38.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:38:38.112 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:38:38.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:38:38.115 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T17:40:15.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6338879836495644) > weight (0.25) 2021-02-20T17:40:15.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.397775419517157) > weight (0.25) 2021-02-20T17:40:15.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6270137555468599) > weight (0.25) 2021-02-20T17:40:15.151 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:40:15.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:40:15.152 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:40:15.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:40:15.148+0000 7ff317f10700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81334) UID: 1000 2021-02-20T17:40:15.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:40:16.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62252f00 [3/3a7a8fa6-6a87-4c14-b1a5-91640b3311d6] start: already running 2021-02-20T17:40:16.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62254580 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T17:40:16.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253180 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T17:40:16.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253400 [3/88d216b6-23a4-48cd-b16a-60ac25aa42b0] start: already running 2021-02-20T17:40:16.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f664ec000 [3/90d8a604-9cc2-4227-8cf3-7e7ff6b73bd0] start: already running 2021-02-20T17:40:16.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62255c00 [3/9a469893-68d6-4d91-a371-3899fcc47454] start: already running 2021-02-20T17:40:16.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253900 [3/b5d77140-8b49-4567-b4bc-f2d77cea0806] start: already running 2021-02-20T17:40:16.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253e00 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T17:40:16.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253680 [3/d9d942d6-60aa-4cd8-a77e-61392adbcc0a] start: already running 2021-02-20T17:40:16.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65218b700 -1 rbd::mirror::ImageReplayer: 0x560f62253b80 [3/df56f5ae-1e1f-4ee7-a8bb-52200231c151] start: already running 2021-02-20T17:40:16.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:40:16.448+0000 7fb65198a700 -1 rbd::mirror::ImageReplayer: 0x560f62f72f80 [3/fc56e3ca-8114-4f4e-9bba-183b58b29c89] start: already running 2021-02-20T17:40:17.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6177023406333979) > weight (0.25) 2021-02-20T17:40:17.508 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:40:17.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:40:17.508 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:40:17.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T17:40:17.504+0000 7fcabf2dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81528) UID: 1000 2021-02-20T17:40:17.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:40:18.153 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:40:18.153 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:40:18.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:40:18.154 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:40:18.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:40:18.155 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:40:18.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:40:18.158 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T17:40:22.509 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:40:22.509 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:40:22.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:40:22.510 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:40:22.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:40:22.511 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:40:22.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:40:22.514 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:41:23.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29712811820869167) > weight (0.25) 2021-02-20T17:41:23.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.54670520998498) > weight (0.25) 2021-02-20T17:41:23.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7482440575991827) > weight (0.25) 2021-02-20T17:41:23.523 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:41:23.523 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:41:23.524 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:41:23.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:41:23.519+0000 7fadbd9fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81437) UID: 1000 2021-02-20T17:41:23.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:41:24.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T17:41:24.787+0000 7fe43fe7e700 -1 rbd::mirror::ImageReplayer: 0x5642a2081e00 [3/ab12bef3-17c2-4b40-8ca2-373b5912a2f4] start: already running 2021-02-20T17:41:29.525 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:41:29.530 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:41:29.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:41:29.531 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:41:29.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:41:29.532 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:41:29.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:41:29.535 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T17:41:35.827 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:41:35.823+0000 7f803c4bb700 -1 reset not still connected to 0x55acc13378e0 2021-02-20T17:41:35.828 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:41:35.823+0000 7f803c4bb700 -1 reset not still connected to 0x55accab01930 2021-02-20T17:42:06.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7841098144781977) > weight (0.25) 2021-02-20T17:42:06.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7778646175689571) > weight (0.25) 2021-02-20T17:42:06.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6967236084861882) > weight (0.25) 2021-02-20T17:42:06.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8805379141645319) > weight (0.25) 2021-02-20T17:42:06.169 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T17:42:31.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2799778836608079) > weight (0.25) 2021-02-20T17:42:31.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5916798738550374) > weight (0.25) 2021-02-20T17:42:31.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9748403707583801) > weight (0.25) 2021-02-20T17:42:31.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.41669051120137446) > weight (0.25) 2021-02-20T17:42:31.537 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T17:43:37.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8450703614127504) > weight (0.25) 2021-02-20T17:43:37.547 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T17:43:37.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T17:43:37.548 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:43:37.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T17:43:37.548+0000 7f0209644700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81813) UID: 1000 2021-02-20T17:43:37.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:43:38.550 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T17:43:38.550 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:43:38.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:43:38.551 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T17:43:38.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T17:43:38.552 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T17:43:38.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T17:43:38.555 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:43:45.170 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:43:45.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:43:45.171 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:43:45.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:43:45.168+0000 7fb65298c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81045) UID: 1000 2021-02-20T17:43:45.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:43:48.172 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:43:48.173 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:43:48.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:43:48.173 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:43:48.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:43:48.174 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:43:48.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:43:48.177 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:44:48.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8782019671704312) > weight (0.25) 2021-02-20T17:44:48.563 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.32711732805111016) > weight (0.25) 2021-02-20T17:44:48.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30369556227454453) > weight (0.25) 2021-02-20T17:44:48.564 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:44:48.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:44:48.565 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:44:48.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:44:48.563+0000 7fb367280700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 81913) UID: 1000 2021-02-20T17:44:48.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:44:49.178 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:44:49.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:44:49.179 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:44:49.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:44:49.178+0000 7f4b8417a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82096) UID: 1000 2021-02-20T17:44:49.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:44:51.566 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:44:51.567 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:44:51.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:44:51.568 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:44:51.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:44:51.568 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:44:51.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:44:51.571 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T17:44:58.180 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:44:58.180 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:44:58.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:44:58.181 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:44:58.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:44:58.181 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:44:58.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:44:58.191 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T17:45:58.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6905642139153704) > weight (0.25) 2021-02-20T17:45:58.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4365824451254624) > weight (0.25) 2021-02-20T17:45:58.193 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:45:58.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:45:58.194 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:45:58.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:45:58.193+0000 7f89bc2f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 81620) UID: 1000 2021-02-20T17:45:58.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:46:05.195 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:46:05.195 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:46:05.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:46:05.196 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:46:05.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:46:05.197 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:46:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:46:05.200 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T17:46:06.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4264333665456984) > weight (0.25) 2021-02-20T17:46:06.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8556144647262847) > weight (0.25) 2021-02-20T17:46:06.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8425639830802832) > weight (0.25) 2021-02-20T17:46:06.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8936172839676514) > weight (0.25) 2021-02-20T17:46:06.573 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T17:46:06.648 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.644+0000 7f508e15e700 -1 reset not still connected to 0x55ac8010b670 2021-02-20T17:46:06.651 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.644+0000 7f508e15e700 -1 reset not still connected to 0x55ac8010b740 2021-02-20T17:46:06.653 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.648+0000 7f508e15e700 -1 reset not still connected to 0x55ac8549a1a0 2021-02-20T17:46:06.653 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.648+0000 7f508e15e700 -1 reset not still connected to 0x55ac862cb8e0 2021-02-20T17:46:06.660 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.648+0000 7f508e15e700 -1 reset not still connected to 0x55ac8653ed00 2021-02-20T17:46:06.660 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.648+0000 7f508e15e700 -1 reset not still connected to 0x55ac87198750 2021-02-20T17:46:06.660 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:46:06.648+0000 7f508e15e700 -1 reset not still connected to 0x55ac8a6b2dd0 2021-02-20T17:47:22.201 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48831781366630533) > weight (0.25) 2021-02-20T17:47:22.210 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:47:22.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:47:22.212 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T17:47:22.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:47:22.207+0000 7fc9e1336700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80571) UID: 1000 2021-02-20T17:47:22.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:47:23.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:47:23.503+0000 7f4726f66700 -1 rbd::mirror::ImageReplayer: 0x55b0203acc80 [3/081dcf2b-6aec-4ae9-b0f2-90a8a2eab556] start: already running 2021-02-20T17:47:23.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:47:23.503+0000 7f4726f66700 -1 rbd::mirror::ImageReplayer: 0x55b0203acf00 [3/0e75152c-6eef-45d5-9c9a-1ea3283e6cd4] start: already running 2021-02-20T17:47:23.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:47:23.503+0000 7f4726f66700 -1 rbd::mirror::ImageReplayer: 0x55b0203ad180 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T17:47:23.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:47:23.503+0000 7f4726765700 -1 rbd::mirror::ImageReplayer: 0x55b0203ad680 [3/4d947326-1304-4f75-9cb5-1cc9895ee2c8] start: already running 2021-02-20T17:47:23.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:47:23.503+0000 7f4726765700 -1 rbd::mirror::ImageReplayer: 0x55b0203ad900 [3/5d895514-421a-41d1-8f87-96dea12501c8] start: already running 2021-02-20T17:47:23.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.511+0000 7fa1656ef700 -1 rbd::mirror::ImageReplayer: 0x5654a2340f00 [3/3eb987d9-c4c6-4abb-b361-cc4a1ae54684] start: already running 2021-02-20T17:47:23.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.511+0000 7fa1656ef700 -1 rbd::mirror::ImageReplayer: 0x5654a64bd180 [3/3f753aad-e91c-4581-b56d-0bf29964e7db] start: already running 2021-02-20T17:47:23.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.511+0000 7fa1656ef700 -1 rbd::mirror::ImageReplayer: 0x5654a64bd900 [3/7726950b-1691-4f9e-8893-d6329ce24b59] start: already running 2021-02-20T17:47:23.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.515+0000 7fa1656ef700 -1 rbd::mirror::ImageReplayer: 0x5654a2341400 [3/b7d39708-c03f-469a-a6fa-228e371ae7a3] start: already running 2021-02-20T17:47:23.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.515+0000 7fa164eee700 -1 rbd::mirror::ImageReplayer: 0x5654a2341b80 [3/c5428c9a-3b56-4be4-96e7-515eb901b226] start: already running 2021-02-20T17:47:23.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:47:23.515+0000 7fa164eee700 -1 rbd::mirror::ImageReplayer: 0x5654a2341900 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T17:47:23.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d9e1d680 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T17:47:23.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d9e1cf00 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T17:47:23.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93180 [3/3738c109-5122-450d-aad4-0ba00b069fb4] start: already running 2021-02-20T17:47:23.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93400 [3/3ca6eed3-b4cc-42a3-8669-fd637c7a8d69] start: already running 2021-02-20T17:47:23.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93680 [3/4c4f4929-06c7-4203-81c6-a0c9560ab291] start: already running 2021-02-20T17:47:23.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d9e1d900 [3/75792560-55ec-454a-bb3c-f7a23e6221d8] start: already running 2021-02-20T17:47:23.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d74f1c700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93b80 [3/7cc83948-bf26-4ed7-a142-2db0f2786e6e] start: already running 2021-02-20T17:47:23.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d7571d700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93900 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T17:47:23.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d7571d700 -1 rbd::mirror::ImageReplayer: 0x55f4d9e1d180 [3/b5d77140-8b49-4567-b4bc-f2d77cea0806] start: already running 2021-02-20T17:47:23.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:47:23.515+0000 7f8d7571d700 -1 rbd::mirror::ImageReplayer: 0x55f4d5d93e00 [3/e450489d-07e5-4769-bd71-337efe806c96] start: already running 2021-02-20T17:47:25.574 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:47:25.582 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:47:25.582 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T17:47:25.582 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:47:25.582 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:47:25.583 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:47:25.579+0000 7f4b75946700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81243) UID: 1000 2021-02-20T17:47:25.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:47:28.213 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:47:28.213 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:47:28.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:47:28.214 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:47:28.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:47:28.215 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:47:28.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:47:28.224 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T17:47:31.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:47:31.685 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:47:31.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:47:31.686 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:47:31.688 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:47:31.689 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-02-20T17:49:16.225 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:49:16.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:49:16.236 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T17:49:16.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:49:16.233+0000 7f8d75f1e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82286) UID: 1000 2021-02-20T17:49:16.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:49:17.236 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:49:17.237 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:49:17.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:49:17.238 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:49:17.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:49:17.239 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:49:17.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:49:17.242 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T17:49:21.690 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:49:21.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:49:21.691 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:49:21.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:49:21.689+0000 7f9217a9c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82580) UID: 1000 2021-02-20T17:49:22.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:49:24.692 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:49:24.693 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:49:24.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:49:24.693 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:49:24.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:49:24.694 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:49:24.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:49:24.697 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:50:34.698 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2596857754264257) > weight (0.25) 2021-02-20T17:50:34.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8994778581075153) > weight (0.25) 2021-02-20T17:50:34.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2637006238755657) > weight (0.25) 2021-02-20T17:50:34.711 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T17:50:34.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T17:50:34.713 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:50:34.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T17:50:34.711+0000 7efd64cfd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 82195) UID: 1000 2021-02-20T17:50:35.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:50:42.714 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T17:50:42.718 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:50:42.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:50:42.719 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T17:50:42.719 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T17:50:42.720 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T17:50:42.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T17:50:42.723 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T17:50:49.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6419925775036391) > weight (0.25) 2021-02-20T17:50:49.243 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T17:50:49.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T17:50:49.244 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T17:50:49.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T17:50:49.243+0000 7ff3bca95700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 82489) UID: 1000 2021-02-20T17:50:49.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:50:52.246 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T17:50:52.246 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:50:52.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:50:52.247 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T17:50:52.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T17:50:52.248 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T17:50:52.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T17:50:52.251 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-02-20T17:51:09.599 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:51:09.593+0000 7f508e15e700 -1 reset not still connected to 0x55ac8dcb76c0 2021-02-20T17:51:57.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7175761529301178) > weight (0.25) 2021-02-20T17:51:57.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7049914846410281) > weight (0.25) 2021-02-20T17:51:57.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9837356746696293) > weight (0.25) 2021-02-20T17:51:57.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.30813545818923893) > weight (0.25) 2021-02-20T17:51:57.262 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T17:52:10.725 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:52:10.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:52:10.733 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:52:10.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:52:10.729+0000 7fbfda3b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82780) UID: 1000 2021-02-20T17:52:11.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:52:18.734 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:52:18.741 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:52:18.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:52:18.742 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:52:18.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:52:18.743 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:52:18.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:52:18.746 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T17:53:06.263 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:53:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:53:06.276 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:53:06.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:53:06.272+0000 7f31bed86700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82681) UID: 1000 2021-02-20T17:53:06.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:53:13.278 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:53:13.283 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:53:13.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:53:13.284 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:53:13.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:53:13.285 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:53:13.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:53:13.288 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T17:53:32.747 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5775971651344685) > weight (0.25) 2021-02-20T17:53:32.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4057868098537383) > weight (0.25) 2021-02-20T17:53:32.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9293192375820177) > weight (0.25) 2021-02-20T17:53:32.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4966202427193456) > weight (0.25) 2021-02-20T17:53:32.749 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T17:54:36.289 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9440423590393985) > weight (0.25) 2021-02-20T17:54:36.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8614924433947357) > weight (0.25) 2021-02-20T17:54:36.290 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T17:54:36.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T17:54:36.292 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T17:54:36.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T17:54:36.290+0000 7f4727767700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 82388) UID: 1000 2021-02-20T17:54:36.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:54:41.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T17:54:41.298 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:54:41.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:54:41.299 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T17:54:41.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T17:54:41.299 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T17:54:41.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T17:54:41.303 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T17:54:54.750 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:54:54.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:54:54.758 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T17:54:54.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:54:54.753+0000 7f47bfc03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83063) UID: 1000 2021-02-20T17:54:55.160 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:55:02.760 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:55:02.771 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:55:02.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:55:02.773 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:55:02.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:55:02.774 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:55:02.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:55:02.777 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T17:55:42.303 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T17:55:42.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T17:55:42.305 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T17:55:42.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T17:55:42.300+0000 7f466bb56700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83154) UID: 1000 2021-02-20T17:55:42.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:55:49.306 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T17:55:49.307 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:55:49.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:55:49.307 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T17:55:49.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T17:55:49.308 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T17:55:49.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T17:55:49.311 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T17:56:51.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7367012558440976) > weight (0.25) 2021-02-20T17:56:51.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.36192730008546226) > weight (0.25) 2021-02-20T17:56:51.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5382365004601386) > weight (0.25) 2021-02-20T17:56:51.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4123993707927721) > weight (0.25) 2021-02-20T17:56:51.780 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T17:56:59.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6370215408265837) > weight (0.25) 2021-02-20T17:56:59.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3021170159451887) > weight (0.25) 2021-02-20T17:56:59.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5067939760227722) > weight (0.25) 2021-02-20T17:56:59.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36624879668848476) > weight (0.25) 2021-02-20T17:56:59.313 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T17:57:54.781 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T17:57:54.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T17:57:54.783 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T17:57:54.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T17:57:54.778+0000 7fcd10777700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83356) UID: 1000 2021-02-20T17:57:55.184 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:58:03.784 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T17:58:03.784 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:58:03.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:58:03.785 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T17:58:03.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T17:58:03.786 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T17:58:03.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T17:58:03.789 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T17:58:44.314 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8517129000669544) > weight (0.25) 2021-02-20T17:58:44.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7123104461926797) > weight (0.25) 2021-02-20T17:58:44.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4823332412365384) > weight (0.25) 2021-02-20T17:58:44.315 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T17:58:44.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T17:58:44.317 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T17:58:44.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T17:58:44.312+0000 7fa165ef0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81722) UID: 1000 2021-02-20T17:58:44.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T17:58:46.075 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T17:58:46.070+0000 7f508e15e700 -1 reset not still connected to 0x55ac8dcb9400 2021-02-20T17:58:48.317 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T17:58:48.318 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T17:58:48.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T17:58:48.319 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T17:58:48.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T17:58:48.320 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T17:58:48.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T17:58:48.323 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-02-20T17:59:17.728 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:59:17.720+0000 7f803c4bb700 -1 reset not still connected to 0x55acb06b2410 2021-02-20T17:59:17.728 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:59:17.720+0000 7f803c4bb700 -1 reset not still connected to 0x55acd5909110 2021-02-20T17:59:17.737 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T17:59:17.732+0000 7f803c4bb700 -1 reset not still connected to 0x55acd590af20 2021-02-20T17:59:55.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5679193642084601) > weight (0.25) 2021-02-20T17:59:55.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.966898373698553) > weight (0.25) 2021-02-20T17:59:55.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4234080191406444) > weight (0.25) 2021-02-20T17:59:55.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7432434606905541) > weight (0.25) 2021-02-20T17:59:55.792 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T18:00:44.324 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6010359498378707) > weight (0.25) 2021-02-20T18:00:44.325 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:00:44.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:00:44.326 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:00:44.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:00:44.322+0000 7fbe0e508700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 82963) UID: 1000 2021-02-20T18:00:44.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:00:51.327 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:00:51.328 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:00:51.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:00:51.329 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:00:51.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:00:51.331 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:00:51.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:00:51.335 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T18:01:30.793 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:01:30.794 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:01:30.794 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:01:30.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:01:30.793+0000 7fdde4ae5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83548) UID: 1000 2021-02-20T18:01:31.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:01:37.796 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:01:37.796 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:01:37.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:01:37.797 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:01:37.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:01:37.797 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:01:37.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:01:37.800 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T18:01:53.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9316715142426858) > weight (0.25) 2021-02-20T18:01:53.344 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:01:53.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:01:53.345 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:01:53.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:01:53.340+0000 7f9a702df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83740) UID: 1000 2021-02-20T18:01:53.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:01:54.346 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:01:54.346 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:01:54.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:01:54.347 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:01:54.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:01:54.348 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:01:54.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:01:54.351 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T18:02:08.053 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:02:08.048+0000 7f803c4bb700 -1 reset not still connected to 0x55acb06b4cb0 2021-02-20T18:02:08.054 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:02:08.048+0000 7f803c4bb700 -1 reset not still connected to 0x55acb06b55a0 2021-02-20T18:03:17.352 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:03:17.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:03:17.354 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:03:17.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:03:17.351+0000 7f63bcfe5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83448) UID: 1000 2021-02-20T18:03:17.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:03:20.354 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:03:20.355 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:03:20.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:03:20.356 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:03:20.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:03:20.357 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:03:20.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:03:20.360 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T18:03:28.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30831541759213754) > weight (0.25) 2021-02-20T18:03:28.801 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38684148542991015) > weight (0.25) 2021-02-20T18:03:28.802 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:03:28.802 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:03:28.803 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:03:28.803 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T18:03:28.798+0000 7fe440e80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80375) UID: 1000 2021-02-20T18:03:29.204 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:03:30.804 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:03:30.804 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:03:30.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:03:30.805 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:03:30.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:03:30.806 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:03:30.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:03:30.809 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:04:48.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8862991929924089) > weight (0.25) 2021-02-20T18:04:48.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9941107334379254) > weight (0.25) 2021-02-20T18:04:48.362 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:04:48.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:04:48.363 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:04:48.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T18:04:48.361+0000 7f35af209700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 83254) UID: 1000 2021-02-20T18:04:48.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:04:51.363 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:04:51.364 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:04:51.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:04:51.365 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:04:51.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:04:51.365 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:04:51.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:04:51.368 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:05:07.809 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:05:07.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:05:07.811 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:05:07.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:05:07.808+0000 7f9ed084d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83839) UID: 1000 2021-02-20T18:05:08.212 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:05:10.812 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:05:10.812 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:05:10.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:05:10.813 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:05:10.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:05:10.814 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:05:10.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:05:10.817 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T18:06:28.369 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:06:28.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:06:28.371 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:06:28.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:06:28.370+0000 7f56ceb9f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 84030) UID: 1000 2021-02-20T18:06:28.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:06:36.372 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:06:36.372 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:06:36.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:06:36.373 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:06:36.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:06:36.374 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:06:36.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:06:36.377 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T18:07:04.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34639214370867566) > weight (0.25) 2021-02-20T18:07:04.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5495794514382674) > weight (0.25) 2021-02-20T18:07:04.819 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:07:04.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:07:04.820 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:07:04.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T18:07:04.818+0000 7f4db883d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 84129) UID: 1000 2021-02-20T18:07:05.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:07:08.820 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:07:08.821 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:07:08.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:07:08.822 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:07:08.822 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:07:08.822 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:07:08.825 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:07:08.825 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T18:08:06.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5745039942855138) > weight (0.25) 2021-02-20T18:08:06.378 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:08:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:08:06.380 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:08:06.380 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:08:06.381 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:08:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:08:06.376+0000 7f3c016c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83930) UID: 1000 2021-02-20T18:08:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:08:12.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:08:12.483 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:08:12.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:08:12.484 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:08:12.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:08:12.486 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T18:08:47.826 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:08:47.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:08:47.828 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:08:47.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:08:47.823+0000 7f3b40b18700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84323) UID: 1000 2021-02-20T18:08:48.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:08:51.829 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:08:51.834 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:08:51.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:08:51.835 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:08:51.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:08:51.836 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:08:51.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:08:51.839 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T18:09:41.487 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9677577060324811) > weight (0.25) 2021-02-20T18:09:41.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3418873264272765) > weight (0.25) 2021-02-20T18:09:41.488 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:09:41.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:09:41.489 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:09:41.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T18:09:41.486+0000 7f7389ac2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84220) UID: 1000 2021-02-20T18:09:41.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:09:50.490 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:09:50.491 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:09:50.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:09:50.492 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:09:50.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:09:50.492 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:09:50.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:09:50.496 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T18:10:29.839 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:10:29.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:10:29.841 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:10:29.842 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:10:29.837+0000 7f800780a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84704) UID: 1000 2021-02-20T18:10:30.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:10:31.842 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:10:31.843 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:10:31.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:10:31.843 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:10:31.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:10:31.844 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:10:31.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:10:31.847 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T18:10:51.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6629346644830635) > weight (0.25) 2021-02-20T18:10:51.498 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:10:51.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:10:51.499 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:10:51.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:10:51.497+0000 7efeec900700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84604) UID: 1000 2021-02-20T18:10:51.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:10:52.499 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:10:52.500 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:10:52.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:10:52.501 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:10:52.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:10:52.501 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:10:52.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:10:52.504 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-02-20T18:11:31.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3703583630135039) > weight (0.25) 2021-02-20T18:11:31.848 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:11:31.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:11:31.849 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:11:31.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:11:31.844+0000 7f5cc0af7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82005) UID: 1000 2021-02-20T18:11:32.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:11:33.850 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:11:33.850 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:11:33.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:11:33.851 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:11:33.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:11:33.852 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:11:33.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:11:33.854 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T18:12:23.505 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5703937052109835) > weight (0.25) 2021-02-20T18:12:23.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2789356668183812) > weight (0.25) 2021-02-20T18:12:23.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8273916741117624) > weight (0.25) 2021-02-20T18:12:23.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2652591782687398) > weight (0.25) 2021-02-20T18:12:23.507 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T18:12:52.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6790978537748799) > weight (0.25) 2021-02-20T18:12:52.856 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6423376283253728) > weight (0.25) 2021-02-20T18:12:52.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3266356188613033) > weight (0.25) 2021-02-20T18:12:52.857 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9687522149513228) > weight (0.25) 2021-02-20T18:12:52.857 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T18:13:29.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.44558781682814286) > weight (0.25) 2021-02-20T18:13:29.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.909375426097739) > weight (0.25) 2021-02-20T18:13:29.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6420080903534728) > weight (0.25) 2021-02-20T18:13:29.509 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:13:29.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:13:29.510 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:13:29.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T18:13:29.509+0000 7fd87b30b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 83639) UID: 1000 2021-02-20T18:13:29.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:13:36.511 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:13:36.512 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:13:36.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:13:36.513 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:13:36.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:13:36.514 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:13:36.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:13:36.517 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T18:13:52.859 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8716008447164094) > weight (0.25) 2021-02-20T18:13:52.859 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.372180918880305) > weight (0.25) 2021-02-20T18:13:52.859 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8125860080371717) > weight (0.25) 2021-02-20T18:13:52.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.35672041806549337) > weight (0.25) 2021-02-20T18:13:52.860 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T18:14:55.518 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.35578337937723337) > weight (0.25) 2021-02-20T18:14:55.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9637622161338226) > weight (0.25) 2021-02-20T18:14:55.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.79110664635548) > weight (0.25) 2021-02-20T18:14:55.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.48157478758965155) > weight (0.25) 2021-02-20T18:14:55.520 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T18:15:09.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5319723616503937) > weight (0.25) 2021-02-20T18:15:09.866 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:15:09.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:15:09.867 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:15:09.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:15:09.863+0000 7f2d5c49f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85088) UID: 1000 2021-02-20T18:15:10.268 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:15:17.868 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:15:17.869 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:15:17.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:15:17.870 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:15:17.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:15:17.870 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:15:17.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:15:17.874 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T18:16:36.875 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4268530409974337) > weight (0.25) 2021-02-20T18:16:36.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8665698471887183) > weight (0.25) 2021-02-20T18:16:36.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39806226106897824) > weight (0.25) 2021-02-20T18:16:36.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.27623381332275265) > weight (0.25) 2021-02-20T18:16:36.878 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-02-20T18:16:52.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8051031037373162) > weight (0.25) 2021-02-20T18:16:52.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6797270288895576) > weight (0.25) 2021-02-20T18:16:52.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7708535313060896) > weight (0.25) 2021-02-20T18:16:52.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5456573191152675) > weight (0.25) 2021-02-20T18:16:52.522 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T18:18:33.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9789214186806828) > weight (0.25) 2021-02-20T18:18:33.880 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:18:33.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:18:33.882 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:18:33.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:18:33.879+0000 7fb73d1b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85282) UID: 1000 2021-02-20T18:18:34.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:18:39.883 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:18:39.883 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:18:39.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:18:39.884 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:18:39.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:18:39.885 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:18:39.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:18:39.888 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T18:18:40.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6037787427452315) > weight (0.25) 2021-02-20T18:18:40.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26791150389833074) > weight (0.25) 2021-02-20T18:18:40.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2930075153108245) > weight (0.25) 2021-02-20T18:18:40.524 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T18:18:40.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T18:18:40.525 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:18:40.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T18:18:40.523+0000 7f9a29277700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 85179) UID: 1000 2021-02-20T18:18:40.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:18:48.525 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T18:18:48.526 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:18:48.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:18:48.527 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T18:18:48.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T18:18:48.527 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T18:18:48.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T18:18:48.530 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-02-20T18:20:13.017 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:20:13.013+0000 7f803c4bb700 -1 reset not still connected to 0x55ac9d7ab670 2021-02-20T18:20:13.018 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:20:13.013+0000 7f803c4bb700 -1 reset not still connected to 0x55acc453d520 2021-02-20T18:20:13.019 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:20:13.013+0000 7f803c4bb700 -1 reset not still connected to 0x55acc453e700 2021-02-20T18:20:13.019 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:20:13.013+0000 7f803c4bb700 -1 reset not still connected to 0x55acc9696f20 2021-02-20T18:20:13.020 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:20:13.013+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896f670 2021-02-20T18:20:19.889 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:20:19.890 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:20:19.890 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:20:19.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:20:19.884+0000 7fc5bcb1d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84898) UID: 1000 2021-02-20T18:20:20.292 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:20:20.891 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:20:20.892 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:20:20.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:20:20.893 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:20:20.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:20:20.893 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:20:20.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:20:20.896 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T18:20:40.532 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:20:40.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:20:40.533 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:20:40.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:20:40.528+0000 7f33155c7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 84414) UID: 1000 2021-02-20T18:20:40.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:20:41.898 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:20:41.887+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972d5a0 2021-02-20T18:20:47.534 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:20:47.535 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:20:47.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:20:47.535 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:20:47.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:20:47.536 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:20:47.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:20:47.539 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-02-20T18:21:37.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9648140634168783) > weight (0.25) 2021-02-20T18:21:37.898 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:21:37.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:21:37.899 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:21:37.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:21:37.895+0000 7feadc7c3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85393) UID: 1000 2021-02-20T18:21:38.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:21:40.900 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:21:40.900 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:21:40.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:21:40.901 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:21:40.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:21:40.901 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:21:40.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:21:40.906 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:22:14.540 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:22:14.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:22:14.541 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:22:14.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:22:14.538+0000 7f0458250700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85677) UID: 1000 2021-02-20T18:22:14.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:22:19.542 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:22:19.542 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:22:19.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:22:19.544 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:22:19.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:22:19.545 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:22:19.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:22:19.548 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T18:23:27.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5260763402755533) > weight (0.25) 2021-02-20T18:23:27.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44429967444933083) > weight (0.25) 2021-02-20T18:23:27.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3453513357105913) > weight (0.25) 2021-02-20T18:23:27.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6856328153312583) > weight (0.25) 2021-02-20T18:23:27.920 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T18:24:02.550 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:24:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:24:02.556 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:24:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:24:02.552+0000 7fde45e47700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85868) UID: 1000 2021-02-20T18:24:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:24:07.557 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:24:07.557 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:24:07.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:24:07.558 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:24:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:24:07.559 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:24:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:24:07.562 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:24:29.921 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8959266878435689) > weight (0.25) 2021-02-20T18:24:29.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6648611553427315) > weight (0.25) 2021-02-20T18:24:29.929 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:24:29.930 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:24:29.931 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:24:29.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T18:24:29.927+0000 7f96569a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 84513) UID: 1000 2021-02-20T18:24:30.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:24:30.932 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:24:30.933 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:24:30.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:24:30.933 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:24:30.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:24:30.934 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:24:30.937 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:24:30.937 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T18:25:33.563 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5248895878780653) > weight (0.25) 2021-02-20T18:25:33.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7992547017235433) > weight (0.25) 2021-02-20T18:25:33.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.34729971473383503) > weight (0.25) 2021-02-20T18:25:33.565 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.31799931478284216) > weight (0.25) 2021-02-20T18:25:33.565 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:25:55.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7057752942173835) > weight (0.25) 2021-02-20T18:25:55.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5373376665752458) > weight (0.25) 2021-02-20T18:25:55.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8000133714273291) > weight (0.25) 2021-02-20T18:25:55.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6162317342920292) > weight (0.25) 2021-02-20T18:25:55.939 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:26:43.013 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:26:43.006+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972d330 2021-02-20T18:26:43.014 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:26:43.010+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972d400 2021-02-20T18:26:43.014 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:26:43.010+0000 7f508e15e700 -1 reset not still connected to 0x55ac8a62a750 2021-02-20T18:27:10.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8898745183046064) > weight (0.25) 2021-02-20T18:27:10.567 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7844814290256825) > weight (0.25) 2021-02-20T18:27:10.568 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:27:10.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:27:10.569 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:27:10.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T18:27:10.564+0000 7f3dbdc17700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84795) UID: 1000 2021-02-20T18:27:10.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:27:14.569 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:27:14.570 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:27:14.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:27:14.570 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:27:14.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:27:14.571 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:27:14.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:27:14.574 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:27:39.941 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:27:39.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:27:39.942 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:27:39.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:27:39.935+0000 7f69b82db700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 85586) UID: 1000 2021-02-20T18:27:40.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:27:46.943 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:27:46.943 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:27:46.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:27:46.944 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:27:46.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:27:46.945 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:27:46.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:27:46.948 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-02-20T18:28:36.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9707533028636814) > weight (0.25) 2021-02-20T18:28:36.585 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6743048551799395) > weight (0.25) 2021-02-20T18:28:36.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8630703651595267) > weight (0.25) 2021-02-20T18:28:36.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9065798529119936) > weight (0.25) 2021-02-20T18:28:36.586 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T18:29:14.949 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:29:14.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:29:14.957 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:29:14.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:29:14.953+0000 7fb88c4cf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86261) UID: 1000 2021-02-20T18:29:15.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:29:23.958 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:29:23.969 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:29:23.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:29:23.970 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:29:23.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:29:23.971 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:29:23.974 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:29:23.974 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:30:31.203 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:30:31.197+0000 7f508e15e700 -1 reset not still connected to 0x55ac8d109260 2021-02-20T18:30:34.587 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7379656341874864) > weight (0.25) 2021-02-20T18:30:34.588 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:30:34.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:30:34.589 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:30:34.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:30:34.583+0000 7fc71a96c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 84989) UID: 1000 2021-02-20T18:30:34.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:30:41.590 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:30:41.590 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:30:41.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:30:41.591 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:30:41.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:30:41.592 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:30:41.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:30:41.595 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T18:31:03.559 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:31:03.552+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972d5a0 2021-02-20T18:31:03.588 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T18:31:03.580+0000 7f62c7ffc700 -1 reset not still connected to 0x555b464efc20 2021-02-20T18:31:03.589 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T18:31:03.580+0000 7f62c7ffc700 -1 reset not still connected to 0x555b4e424dd0 2021-02-20T18:31:03.589 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T18:31:03.580+0000 7f62c7ffc700 -1 reset not still connected to 0x555b560a61a0 2021-02-20T18:31:03.589 INFO:tasks.ceph.cluster1.osd.1.smithi106.stderr:2021-02-20T18:31:03.580+0000 7f62c7ffc700 -1 reset not still connected to 0x555b560a8220 2021-02-20T18:31:07.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39142505220949164) > weight (0.25) 2021-02-20T18:31:07.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8144093899662465) > weight (0.25) 2021-02-20T18:31:07.975 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7589868693655122) > weight (0.25) 2021-02-20T18:31:07.976 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:31:07.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:31:07.976 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T18:31:07.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T18:31:07.974+0000 7f5ea6d87700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 82872) UID: 1000 2021-02-20T18:31:08.378 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:31:13.978 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:31:13.978 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:31:13.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:31:13.979 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:31:13.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:31:13.979 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:31:13.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:31:13.982 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T18:32:36.596 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:32:36.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:32:36.598 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:32:36.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:32:36.593+0000 7f70e1d4b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85967) UID: 1000 2021-02-20T18:32:36.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:32:39.983 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:32:39.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:32:39.984 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:32:39.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:32:39.976+0000 7f4af8c14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86353) UID: 1000 2021-02-20T18:32:40.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:32:41.598 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:32:41.599 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:32:41.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:32:41.600 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:32:41.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:32:41.600 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:32:41.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:32:41.603 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T18:32:44.985 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:32:44.985 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:32:44.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:32:44.986 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:32:44.986 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:32:44.987 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:32:44.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:32:44.990 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T18:32:49.512 INFO:tasks.ceph.cluster2.osd.1.smithi114.stderr:2021-02-20T18:32:49.504+0000 7fc273bc1700 -1 reset not still connected to 0x561183ab1860 2021-02-20T18:32:51.502 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:32:51.498+0000 7f508e15e700 -1 reset not still connected to 0x55ac86a5b6c0 2021-02-20T18:34:00.605 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:34:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:34:00.606 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:34:00.606 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:34:00.607 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:34:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:34:00.603+0000 7f7f03746700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86638) UID: 1000 2021-02-20T18:34:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:34:06.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:34:06.708 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:34:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:34:06.709 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:34:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:34:06.712 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T18:34:29.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6031976120324702) > weight (0.25) 2021-02-20T18:34:29.991 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8977802091780648) > weight (0.25) 2021-02-20T18:34:29.991 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8593757898109903) > weight (0.25) 2021-02-20T18:34:29.992 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3605054745674555) > weight (0.25) 2021-02-20T18:34:29.992 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T18:35:12.713 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:35:12.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:35:12.714 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:35:12.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:35:12.709+0000 7fa9ba6e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86828) UID: 1000 2021-02-20T18:35:13.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:35:14.716 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:35:14.716 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:35:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:35:14.717 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:35:14.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:35:14.717 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:35:14.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:35:14.721 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T18:35:36.993 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:35:37.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:35:37.002 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:35:37.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:35:36.997+0000 7f54f3f69700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86729) UID: 1000 2021-02-20T18:35:37.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:35:42.003 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:35:42.004 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:35:42.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:35:42.004 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:35:42.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:35:42.005 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:35:42.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:35:42.008 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T18:36:20.722 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:36:20.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:36:20.732 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:36:20.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:36:20.724+0000 7f798c20d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86930) UID: 1000 2021-02-20T18:36:21.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:36:27.733 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:36:27.733 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:36:27.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:36:27.734 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:36:27.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:36:27.735 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:36:27.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:36:27.737 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T18:36:32.665 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:36:32.656+0000 7f508e15e700 -1 reset not still connected to 0x55ac8067dc70 2021-02-20T18:36:43.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3167648770540197) > weight (0.25) 2021-02-20T18:36:43.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.33111977893296674) > weight (0.25) 2021-02-20T18:36:43.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6126870498071686) > weight (0.25) 2021-02-20T18:36:43.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.42951263312358146) > weight (0.25) 2021-02-20T18:36:43.016 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T18:37:45.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9097820498987802) > weight (0.25) 2021-02-20T18:37:45.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.621286920133167) > weight (0.25) 2021-02-20T18:37:45.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8504594244254904) > weight (0.25) 2021-02-20T18:37:45.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4258341971811058) > weight (0.25) 2021-02-20T18:37:45.740 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-02-20T18:38:29.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7918929493706913) > weight (0.25) 2021-02-20T18:38:29.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5175298339610395) > weight (0.25) 2021-02-20T18:38:29.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3155089538395659) > weight (0.25) 2021-02-20T18:38:29.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8302838543180762) > weight (0.25) 2021-02-20T18:38:29.018 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-02-20T18:39:24.741 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:39:24.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:39:24.743 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:39:24.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:39:24.736+0000 7f9f069dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87121) UID: 1000 2021-02-20T18:39:25.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:39:25.744 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:39:25.744 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:39:25.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:39:25.745 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:39:25.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:39:25.746 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:39:25.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:39:25.749 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-02-20T18:40:02.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8084834886428135) > weight (0.25) 2021-02-20T18:40:02.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7589519069862865) > weight (0.25) 2021-02-20T18:40:02.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6148380123975079) > weight (0.25) 2021-02-20T18:40:02.033 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6809515115076212) > weight (0.25) 2021-02-20T18:40:02.033 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T18:40:29.750 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:40:29.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:40:29.762 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T18:40:29.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:40:29.758+0000 7f467dce3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87222) UID: 1000 2021-02-20T18:40:29.809 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:40:29.803+0000 7f508e15e700 -1 reset not still connected to 0x55ac8b9f2220 2021-02-20T18:40:29.810 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:40:29.803+0000 7f508e15e700 -1 reset not still connected to 0x55ac916e3dc0 2021-02-20T18:40:29.810 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:40:29.803+0000 7f508e15e700 -1 reset not still connected to 0x55ac93cb4b60 2021-02-20T18:40:29.810 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:40:29.803+0000 7f508e15e700 -1 reset not still connected to 0x55ac93cb63c0 2021-02-20T18:40:29.811 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:40:29.803+0000 7f508e15e700 -1 reset not still connected to 0x55ac93cb6a40 2021-02-20T18:40:30.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:40:37.763 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:40:37.763 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:40:37.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:40:37.764 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:40:37.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:40:37.764 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:40:37.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:40:37.767 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T18:41:13.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5958549885389597) > weight (0.25) 2021-02-20T18:41:13.034 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43489444433211744) > weight (0.25) 2021-02-20T18:41:13.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42415231811545195) > weight (0.25) 2021-02-20T18:41:13.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.442139290640187) > weight (0.25) 2021-02-20T18:41:13.035 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T18:42:24.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5515226795728663) > weight (0.25) 2021-02-20T18:42:24.769 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38907602020926557) > weight (0.25) 2021-02-20T18:42:24.769 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:42:24.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:42:24.770 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:42:24.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T18:42:24.764+0000 7f4c8ef2e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 86162) UID: 1000 2021-02-20T18:42:25.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:42:26.771 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:42:26.771 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:42:26.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:42:26.772 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:42:26.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:42:26.773 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:42:26.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:42:26.778 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T18:42:51.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3904270051964025) > weight (0.25) 2021-02-20T18:42:51.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.26377873474046654) > weight (0.25) 2021-02-20T18:42:51.037 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T18:42:51.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T18:42:51.038 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:42:51.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T18:42:51.035+0000 7fddc55ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 86066) UID: 1000 2021-02-20T18:42:51.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:42:53.039 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T18:42:53.039 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:42:53.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:42:53.040 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T18:42:53.041 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T18:42:53.041 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T18:42:53.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T18:42:53.044 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T18:42:56.978 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:42:56.967+0000 7f803c4bb700 -1 reset not still connected to 0x55acc453fa80 2021-02-20T18:43:42.779 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.783591098989318) > weight (0.25) 2021-02-20T18:43:42.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7369646326303213) > weight (0.25) 2021-02-20T18:43:42.786 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4953322323993329) > weight (0.25) 2021-02-20T18:43:42.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5941334930109811) > weight (0.25) 2021-02-20T18:43:42.787 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T18:44:42.045 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:44:42.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:44:42.047 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T18:44:42.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:44:42.041+0000 7ff58861e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87030) UID: 1000 2021-02-20T18:44:42.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:44:44.048 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:44:44.049 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:44:44.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:44:44.049 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:44:44.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:44:44.050 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:44:44.053 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:44:44.053 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:44:49.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3555434919938327) > weight (0.25) 2021-02-20T18:44:49.789 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:44:49.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:44:49.789 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:44:49.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:44:49.785+0000 7f4f1d39a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 86446) UID: 1000 2021-02-20T18:44:50.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:44:53.790 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:44:53.791 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:44:53.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:44:53.791 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:44:53.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:44:53.792 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:44:53.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:44:53.795 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T18:46:06.054 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8189081739323006) > weight (0.25) 2021-02-20T18:46:06.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6029556126667114) > weight (0.25) 2021-02-20T18:46:06.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9957156114238623) > weight (0.25) 2021-02-20T18:46:06.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6395048569623323) > weight (0.25) 2021-02-20T18:46:06.063 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T18:46:10.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9563593654657031) > weight (0.25) 2021-02-20T18:46:10.803 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:46:10.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:46:10.805 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:46:10.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:46:10.795+0000 7f6ebab12700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87772) UID: 1000 2021-02-20T18:46:11.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:46:17.805 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:46:17.806 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:46:17.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:46:17.807 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:46:17.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:46:17.807 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:46:17.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:46:17.811 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-02-20T18:47:19.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42457598338337676) > weight (0.25) 2021-02-20T18:47:19.066 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:47:19.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:47:19.067 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:47:19.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:47:19.061+0000 7f6d5c17d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85777) UID: 1000 2021-02-20T18:47:19.468 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:47:22.067 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:47:22.068 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:47:22.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:47:22.069 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:47:22.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:47:22.070 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:47:22.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:47:22.073 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-02-20T18:47:24.812 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:47:24.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:47:24.813 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:47:24.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:47:24.805+0000 7fe341c37700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87322) UID: 1000 2021-02-20T18:47:25.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:47:31.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:47:31.814 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:47:31.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:47:31.815 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:47:31.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:47:31.815 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:47:31.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:47:31.818 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T18:47:57.411 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:47:57.405+0000 7f803c4bb700 -1 reset not still connected to 0x55acd590b330 2021-02-20T18:47:57.412 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:47:57.405+0000 7f803c4bb700 -1 reset not still connected to 0x55acdacf5ee0 2021-02-20T18:48:33.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46851389270652066) > weight (0.25) 2021-02-20T18:48:33.820 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7827260326192387) > weight (0.25) 2021-02-20T18:48:33.820 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T18:48:33.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T18:48:33.821 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:48:33.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T18:48:33.812+0000 7fadb72aa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 87422) UID: 1000 2021-02-20T18:48:34.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:48:37.074 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:48:37.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:48:37.075 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:48:37.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:48:37.072+0000 7f11d989f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87681) UID: 1000 2021-02-20T18:48:37.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:48:38.075 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:48:38.076 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:48:38.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:48:38.077 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:48:38.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:48:38.077 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:48:38.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:48:38.080 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T18:48:42.822 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T18:48:42.823 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:48:42.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:48:42.823 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T18:48:42.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T18:48:42.824 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T18:48:42.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T18:48:42.827 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T18:49:46.082 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:49:46.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:49:46.083 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T18:49:46.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:49:46.078+0000 7fcb6e5b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88166) UID: 1000 2021-02-20T18:49:46.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:49:49.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:49:49.085 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:49:49.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:49:49.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:49:49.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:49:49.087 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:49:49.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:49:49.091 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T18:50:04.828 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9889881686050445) > weight (0.25) 2021-02-20T18:50:04.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5150193779665145) > weight (0.25) 2021-02-20T18:50:04.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.689737236122562) > weight (0.25) 2021-02-20T18:50:04.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.779903696916595) > weight (0.25) 2021-02-20T18:50:04.830 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-02-20T18:50:17.668 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T18:50:17.662+0000 7f803c4bb700 -1 reset not still connected to 0x55acc9697740 2021-02-20T18:51:19.091 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9138577623984021) > weight (0.25) 2021-02-20T18:51:19.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4220337261317255) > weight (0.25) 2021-02-20T18:51:19.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5424488890863072) > weight (0.25) 2021-02-20T18:51:19.093 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T18:51:19.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T18:51:19.094 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:51:19.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T18:51:19.088+0000 7f339c84e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 86545) UID: 1000 2021-02-20T18:51:19.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:23.094 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T18:51:23.095 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:23.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:23.096 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T18:51:23.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T18:51:23.096 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T18:51:23.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T18:51:23.099 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T18:51:44.830 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:51:44.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:51:44.831 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:51:44.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:51:44.828+0000 7fb2076ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88065) UID: 1000 2021-02-20T18:51:45.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:51:49.833 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:51:49.834 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:51:49.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:51:49.835 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:51:49.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:51:49.836 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:51:49.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:51:49.841 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T18:52:19.364 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T18:52:19.358+0000 7f508e15e700 -1 reset not still connected to 0x55ac8647c5b0 2021-02-20T18:52:53.100 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:52:53.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:52:53.109 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:52:53.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:52:53.102+0000 7f841de24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88356) UID: 1000 2021-02-20T18:52:53.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:53:02.110 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:53:02.633 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:53:02.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:53:02.634 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:53:02.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:53:02.635 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:53:02.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:53:02.640 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T18:53:44.842 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5992129435497772) > weight (0.25) 2021-02-20T18:53:44.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6822880594569991) > weight (0.25) 2021-02-20T18:53:44.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6931297977367953) > weight (0.25) 2021-02-20T18:53:44.843 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5813828751126637) > weight (0.25) 2021-02-20T18:53:44.843 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:54:27.641 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:54:27.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:54:27.649 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T18:54:27.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:54:27.650 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:54:27.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:54:27.644+0000 7fcf9f4b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88639) UID: 1000 2021-02-20T18:54:28.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:54:33.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:54:33.752 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:54:33.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:54:33.753 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:54:33.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:54:33.755 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-02-20T18:55:21.844 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:55:21.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:55:21.855 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:55:21.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:55:21.851+0000 7f2e9c392700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88539) UID: 1000 2021-02-20T18:55:22.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:55:26.856 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:55:26.856 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:55:26.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:55:26.857 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:55:26.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:55:26.858 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:55:26.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:55:26.864 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T18:55:57.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8147658968483289) > weight (0.25) 2021-02-20T18:55:57.763 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T18:55:57.763 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T18:55:57.764 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T18:55:57.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T18:55:57.758+0000 7f0cf8645700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 87974) UID: 1000 2021-02-20T18:55:58.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:56:02.765 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T18:56:02.765 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:56:02.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:56:02.766 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T18:56:02.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T18:56:02.767 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T18:56:02.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T18:56:02.770 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T18:57:09.865 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9586256289639916) > weight (0.25) 2021-02-20T18:57:09.874 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T18:57:09.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T18:57:09.875 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T18:57:09.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T18:57:09.869+0000 7f0d369fb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87874) UID: 1000 2021-02-20T18:57:10.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:57:13.876 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T18:57:13.876 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:57:13.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:57:13.877 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T18:57:13.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T18:57:13.878 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T18:57:13.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T18:57:13.880 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T18:57:39.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8857039567955792) > weight (0.25) 2021-02-20T18:57:39.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8905718582965906) > weight (0.25) 2021-02-20T18:57:39.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9003283632869701) > weight (0.25) 2021-02-20T18:57:39.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9656076287931717) > weight (0.25) 2021-02-20T18:57:39.772 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-02-20T18:58:16.881 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:58:16.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:58:16.883 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T18:58:16.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:58:16.875+0000 7f282e5fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88825) UID: 1000 2021-02-20T18:58:17.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:58:25.884 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:58:25.885 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:58:25.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:58:25.885 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:58:25.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:58:25.886 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:58:25.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:58:25.889 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T18:58:45.760 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T18:58:45.751+0000 7ff4f84bf700 -1 reset not still connected to 0x55db15e04700 2021-02-20T18:58:45.760 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T18:58:45.751+0000 7ff4f84bf700 -1 reset not still connected to 0x55db15e054d0 2021-02-20T18:58:45.760 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T18:58:45.751+0000 7ff4f84bf700 -1 reset not still connected to 0x55db1e486560 2021-02-20T18:58:45.761 INFO:tasks.ceph.cluster2.osd.0.smithi114.stderr:2021-02-20T18:58:45.751+0000 7ff4f84bf700 -1 reset not still connected to 0x55db665d6dd0 2021-02-20T18:59:23.773 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T18:59:23.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T18:59:23.774 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T18:59:23.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T18:59:23.770+0000 7fdff1ffd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88731) UID: 1000 2021-02-20T18:59:24.176 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:59:30.776 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T18:59:30.776 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:59:30.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:59:30.777 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T18:59:30.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T18:59:30.778 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T18:59:30.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T18:59:30.781 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T18:59:50.890 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T18:59:50.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T18:59:50.898 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T18:59:50.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T18:59:50.893+0000 7f918de79700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89116) UID: 1000 2021-02-20T18:59:51.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T18:59:51.899 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T18:59:51.900 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T18:59:51.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T18:59:51.901 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T18:59:51.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T18:59:51.901 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T18:59:51.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T18:59:51.904 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T19:00:18.755 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:00:18.748+0000 7f803c4bb700 -1 reset not still connected to 0x55acb6940e50 2021-02-20T19:00:23.380 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T19:00:23.375+0000 7f508e15e700 -1 reset not still connected to 0x55ac8010aa40 2021-02-20T19:00:51.782 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:00:51.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:00:51.783 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:00:51.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:00:51.784+0000 7f3d82fa1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89215) UID: 1000 2021-02-20T19:00:52.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:01:00.784 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:01:00.785 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:01:00.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:01:00.786 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:01:00.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:01:00.787 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:01:00.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:01:00.791 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:01:12.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8349816055243605) > weight (0.25) 2021-02-20T19:01:12.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3237772451575769) > weight (0.25) 2021-02-20T19:01:12.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.892830123275156) > weight (0.25) 2021-02-20T19:01:12.916 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45180418855084525) > weight (0.25) 2021-02-20T19:01:12.917 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:02:32.793 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:02:32.804 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:02:32.804 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:02:32.804 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:02:32.805 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:02:32.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:02:32.798+0000 7f00147b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89407) UID: 1000 2021-02-20T19:02:33.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:02:38.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:02:38.907 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:02:38.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:02:38.907 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:02:38.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:02:38.910 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-02-20T19:02:47.917 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8969416897638983) > weight (0.25) 2021-02-20T19:02:47.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4826019252789713) > weight (0.25) 2021-02-20T19:02:47.918 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:02:47.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:02:47.919 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:02:47.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T19:02:47.913+0000 7f0bb6878700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 88257) UID: 1000 2021-02-20T19:02:48.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:02:51.920 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:02:51.926 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:02:51.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:02:51.927 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:02:51.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:02:51.928 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:02:51.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:02:51.931 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-02-20T19:04:03.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8792408257298674) > weight (0.25) 2021-02-20T19:04:03.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7505660293038463) > weight (0.25) 2021-02-20T19:04:03.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6798871137047334) > weight (0.25) 2021-02-20T19:04:03.997 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8911323786628124) > weight (0.25) 2021-02-20T19:04:03.998 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T19:04:27.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8217120042405025) > weight (0.25) 2021-02-20T19:04:27.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.35097810819831676) > weight (0.25) 2021-02-20T19:04:27.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6673899084245961) > weight (0.25) 2021-02-20T19:04:27.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5825885750289663) > weight (0.25) 2021-02-20T19:04:27.935 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-02-20T19:05:15.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9313337657689635) > weight (0.25) 2021-02-20T19:05:16.000 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:05:16.000 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:05:16.002 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:05:16.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:05:15.994+0000 7fd2bdfef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 88924) UID: 1000 2021-02-20T19:05:16.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:05:21.003 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:05:21.007 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:05:21.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:05:21.008 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:05:21.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:05:21.008 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:05:21.011 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:05:21.012 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:06:16.939 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:06:16.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:06:16.943 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:06:16.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:06:16.941+0000 7f4230279700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89307) UID: 1000 2021-02-20T19:06:17.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:06:19.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:06:19.946 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:06:19.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:06:19.947 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:06:19.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:06:19.947 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:06:19.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:06:19.951 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-02-20T19:07:06.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6982343948181086) > weight (0.25) 2021-02-20T19:07:06.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.608178156505965) > weight (0.25) 2021-02-20T19:07:06.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7330047028297658) > weight (0.25) 2021-02-20T19:07:06.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8848457417866453) > weight (0.25) 2021-02-20T19:07:06.023 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T19:07:32.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4277511703857305) > weight (0.25) 2021-02-20T19:07:32.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5358559925120362) > weight (0.25) 2021-02-20T19:07:32.959 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:07:32.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:07:32.960 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:07:32.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T19:07:32.955+0000 7fcfe3738700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 89589) UID: 1000 2021-02-20T19:07:33.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:07:34.961 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:07:34.961 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:07:34.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:07:34.962 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:07:34.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:07:34.963 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:07:34.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:07:34.967 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-02-20T19:08:25.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42923567914268435) > weight (0.25) 2021-02-20T19:08:26.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30187190859722) > weight (0.25) 2021-02-20T19:08:26.508 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:08:26.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:08:26.511 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:08:26.511 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:08:26.511 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:08:26.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:08:26.502+0000 7fe9be92b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 87521) UID: 1000 2021-02-20T19:08:26.911 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:08:32.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:08:32.614 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:08:32.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:08:32.615 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:08:32.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:08:32.618 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:08:52.968 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:08:52.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:08:52.974 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:08:52.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:08:52.965+0000 7fa774ca7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89784) UID: 1000 2021-02-20T19:08:53.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:08:54.975 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:08:54.975 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:08:54.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:08:54.976 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:08:54.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:08:54.977 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:08:54.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:08:54.979 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-02-20T19:09:51.274 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T19:09:51.265+0000 7f508e15e700 -1 reset not still connected to 0x55ac7bac59b0 2021-02-20T19:10:10.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5462701594758045) > weight (0.25) 2021-02-20T19:10:10.632 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:10:10.633 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:10:10.635 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:10:10.635 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:10:10.635 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:10:10.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:10:10.624+0000 7fa287c3e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 89693) UID: 1000 2021-02-20T19:10:11.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:10:16.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:10:16.738 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:10:16.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:10:16.740 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:10:16.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:10:16.742 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-02-20T19:10:24.980 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.904090201319019) > weight (0.25) 2021-02-20T19:10:24.981 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:10:24.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:10:24.982 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:10:24.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T19:10:24.975+0000 7f642cdcb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 89016) UID: 1000 2021-02-20T19:10:25.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:10:27.983 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:10:27.983 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:10:27.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:10:27.984 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:10:27.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:10:27.985 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:10:27.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:10:27.987 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-02-20T19:11:27.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7305504002372747) > weight (0.25) 2021-02-20T19:11:27.744 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39143800961610353) > weight (0.25) 2021-02-20T19:11:27.745 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.45581440535436035) > weight (0.25) 2021-02-20T19:11:27.745 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:11:27.746 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:11:27.746 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:11:27.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:11:27.742+0000 7f4893eda700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 88447) UID: 1000 2021-02-20T19:11:28.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:11:36.747 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:11:36.756 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:11:36.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:11:36.757 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:11:36.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:11:36.758 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:11:36.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:11:36.765 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T19:11:37.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.978791191267767) > weight (0.25) 2021-02-20T19:11:37.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3799837702762764) > weight (0.25) 2021-02-20T19:11:37.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6501222393726847) > weight (0.25) 2021-02-20T19:11:37.990 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.38506704428979976) > weight (0.25) 2021-02-20T19:11:37.990 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-02-20T19:12:48.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35523067421468313) > weight (0.25) 2021-02-20T19:12:48.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8668657050910079) > weight (0.25) 2021-02-20T19:12:48.815 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:12:48.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:12:48.816 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:12:48.816 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:12:48.816 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:12:48.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:12:48.808+0000 7f1db7e46700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 89983) UID: 1000 2021-02-20T19:12:49.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:12:54.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:12:54.919 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:12:54.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:12:54.920 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:12:54.922 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:12:54.922 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:13:14.992 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:13:14.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:13:14.993 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:13:14.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:13:14.984+0000 7fb512ec5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 90074) UID: 1000 2021-02-20T19:13:15.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:13:18.741 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T19:13:18.728+0000 7f508e15e700 -1 reset not still connected to 0x55ac8867acb0 2021-02-20T19:13:18.741 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T19:13:18.728+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972cbe0 2021-02-20T19:13:18.741 INFO:tasks.ceph.cluster1.osd.2.smithi106.stderr:2021-02-20T19:13:18.728+0000 7f508e15e700 -1 reset not still connected to 0x55ac8972d5a0 2021-02-20T19:13:19.993 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:13:19.994 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:13:19.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:13:19.994 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:13:19.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:13:19.995 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:13:19.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:13:19.998 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:14:02.923 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7941675166945898) > weight (0.25) 2021-02-20T19:14:02.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4400950112985692) > weight (0.25) 2021-02-20T19:14:02.924 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:14:02.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:14:02.925 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:14:02.927 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:14:02.919+0000 7f3981169700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 90456) UID: 1000 2021-02-20T19:14:03.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:14:04.926 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:14:04.927 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:14:04.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:14:04.927 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:14:04.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:14:04.928 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:14:04.931 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:14:04.931 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:14:54.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7868147413979064) > weight (0.25) 2021-02-20T19:14:55.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7679215253171277) > weight (0.25) 2021-02-20T19:14:55.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6616218751096358) > weight (0.25) 2021-02-20T19:14:55.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4411143022784807) > weight (0.25) 2021-02-20T19:14:55.007 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:15:26.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8257397912212725) > weight (0.25) 2021-02-20T19:15:26.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3985981474732492) > weight (0.25) 2021-02-20T19:15:26.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.31207657074619655) > weight (0.25) 2021-02-20T19:15:26.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3022329878254011) > weight (0.25) 2021-02-20T19:15:26.933 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T19:16:17.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6937397131371819) > weight (0.25) 2021-02-20T19:16:17.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7570984650323042) > weight (0.25) 2021-02-20T19:16:17.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.639900428861467) > weight (0.25) 2021-02-20T19:16:17.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2664142790614278) > weight (0.25) 2021-02-20T19:16:17.009 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-02-20T19:16:55.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3864756225088847) > weight (0.25) 2021-02-20T19:16:55.938 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:16:55.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:16:55.944 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:16:55.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:16:55.945 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:16:55.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:16:55.939+0000 7f6e60e31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 90174) UID: 1000 2021-02-20T19:16:56.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:17:02.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:17:02.048 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:17:02.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:17:02.049 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:17:02.053 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:17:02.053 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:17:25.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5711003438261796) > weight (0.25) 2021-02-20T19:17:25.011 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.29129020410534323) > weight (0.25) 2021-02-20T19:17:25.012 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:17:25.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:17:25.013 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:17:25.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T19:17:25.014+0000 7f33ffad1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 89883) UID: 1000 2021-02-20T19:17:25.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:17:27.015 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:17:27.015 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:17:27.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:17:27.016 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:17:27.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:17:27.017 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:17:27.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:17:27.019 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-02-20T19:18:40.055 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:18:40.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:18:40.057 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:18:40.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:18:40.053+0000 7f98e13ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89498) UID: 1000 2021-02-20T19:18:40.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:18:41.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:18:41.329+0000 7f1389d74700 -1 rbd::mirror::ImageReplayer: 0x5599c9d1fb80 [3/6a6f5866-70c0-4063-8b45-7c49d8fcdf4b] start: already running 2021-02-20T19:18:41.342 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:18:41.333+0000 7f867fba3700 -1 rbd::mirror::ImageReplayer: 0x561fd405f900 [3/081dcf2b-6aec-4ae9-b0f2-90a8a2eab556] start: already running 2021-02-20T19:18:41.342 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:18:41.333+0000 7f867fba3700 -1 rbd::mirror::ImageReplayer: 0x561fd405fb80 [3/1df8685b-4ef2-4120-ae1d-74963f7c3ef0] start: already running 2021-02-20T19:18:48.058 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:18:48.058 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:18:48.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:18:48.059 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:18:48.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:18:48.060 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:18:48.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:18:48.063 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T19:18:53.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5874718561984454) > weight (0.25) 2021-02-20T19:18:53.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45578445007689294) > weight (0.25) 2021-02-20T19:18:53.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6950822405947852) > weight (0.25) 2021-02-20T19:18:53.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3475005100217433) > weight (0.25) 2021-02-20T19:18:53.021 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T19:19:35.195 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:19:35.183+0000 7f803c4bb700 -1 reset not still connected to 0x55acb4afb8e0 2021-02-20T19:19:37.044 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:19:37.035+0000 7f803c4bb700 -1 reset not still connected to 0x55acc05b48a0 2021-02-20T19:19:37.045 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:19:37.035+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896fb50 2021-02-20T19:19:53.022 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8882014087988954) > weight (0.25) 2021-02-20T19:19:53.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45456470609712907) > weight (0.25) 2021-02-20T19:19:53.029 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.28206403591843987) > weight (0.25) 2021-02-20T19:19:53.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6444201678532052) > weight (0.25) 2021-02-20T19:19:53.030 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-02-20T19:19:55.462 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:19:55.451+0000 7f803c4bb700 -1 reset not still connected to 0x55accf696150 2021-02-20T19:19:55.462 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:19:55.451+0000 7f803c4bb700 -1 reset not still connected to 0x55ace896fb50 2021-02-20T19:20:05.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9141610257881239) > weight (0.25) 2021-02-20T19:20:05.076 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:20:05.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:20:05.078 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:20:05.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:20:05.071+0000 7effd3b60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 90745) UID: 1000 2021-02-20T19:20:05.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:20:08.078 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:20:08.084 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:20:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:20:08.085 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:20:08.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:20:08.085 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:20:08.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:20:08.091 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:21:12.030 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:21:12.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:21:12.033 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:21:12.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:21:12.029+0000 7f0b81449700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 90547) UID: 1000 2021-02-20T19:21:12.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:21:20.033 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:21:20.048 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:21:20.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:21:20.049 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:21:20.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:21:20.050 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:21:20.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:21:20.054 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T19:21:24.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6173433199041118) > weight (0.25) 2021-02-20T19:21:24.093 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9284940213581212) > weight (0.25) 2021-02-20T19:21:24.093 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:21:24.093 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:21:24.094 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:21:24.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:21:24.089+0000 7f8680ba5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 90647) UID: 1000 2021-02-20T19:21:24.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:21:26.094 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:21:26.095 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:21:26.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:21:26.096 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:21:26.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:21:26.096 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:21:26.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:21:26.100 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-02-20T19:22:20.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8942189676850635) > weight (0.25) 2021-02-20T19:22:20.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.33233551645728354) > weight (0.25) 2021-02-20T19:22:20.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6784936021603586) > weight (0.25) 2021-02-20T19:22:20.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.532057923919818) > weight (0.25) 2021-02-20T19:22:20.063 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-02-20T19:22:46.101 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6730543313265723) > weight (0.25) 2021-02-20T19:22:46.101 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:22:46.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:22:46.103 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-02-20T19:22:46.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:22:46.095+0000 7fbb4d393700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 91029) UID: 1000 2021-02-20T19:22:46.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:22:49.103 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:22:49.104 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:22:49.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:22:49.105 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:22:49.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:22:49.105 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:22:49.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:22:49.108 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:23:21.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2516808782725015) > weight (0.25) 2021-02-20T19:23:21.065 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:23:21.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:23:21.066 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:23:21.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T19:23:21.059+0000 7ff5ad0fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 90266) UID: 1000 2021-02-20T19:23:21.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:23:29.067 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:23:29.068 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:23:29.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:23:29.069 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:23:29.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:23:29.070 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:23:29.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:23:29.074 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-02-20T19:24:21.110 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:24:21.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:24:21.118 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:24:21.118 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:24:21.118 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:24:21.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:24:21.109+0000 7f913294e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 90936) UID: 1000 2021-02-20T19:24:21.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:24:27.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:24:27.220 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:24:27.220 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:24:27.221 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:24:27.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:24:27.224 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:25:16.075 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.65258036417067) > weight (0.25) 2021-02-20T19:25:16.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3692649848108086) > weight (0.25) 2021-02-20T19:25:16.089 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:25:16.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:25:16.090 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-02-20T19:25:16.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T19:25:16.084+0000 7f96e5400700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 90837) UID: 1000 2021-02-20T19:25:16.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:25:22.091 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:25:22.092 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:25:22.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:25:22.092 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:25:22.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:25:22.093 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:25:22.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:25:22.096 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-02-20T19:25:29.225 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25526777482234264) > weight (0.25) 2021-02-20T19:25:29.225 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9685906672791862) > weight (0.25) 2021-02-20T19:25:29.226 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37284615132853716) > weight (0.25) 2021-02-20T19:25:29.226 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8687728457249347) > weight (0.25) 2021-02-20T19:25:29.226 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T19:26:54.097 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:26:54.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:26:54.501 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:26:54.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:26:54.494+0000 7f820cb53700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 91122) UID: 1000 2021-02-20T19:26:54.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:27:01.502 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:27:01.503 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:27:01.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:27:01.504 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:27:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:27:01.505 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:27:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:27:01.508 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-02-20T19:27:12.227 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3601547010027504) > weight (0.25) 2021-02-20T19:27:12.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8544236997557659) > weight (0.25) 2021-02-20T19:27:12.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.683720412148328) > weight (0.25) 2021-02-20T19:27:12.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.49748764411399693) > weight (0.25) 2021-02-20T19:27:12.235 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-02-20T19:28:29.236 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:28:29.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:28:29.238 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:28:29.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:28:29.232+0000 7fd3330ee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 91502) UID: 1000 2021-02-20T19:28:29.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:28:30.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:28:30.239 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:28:30.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:28:30.240 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:28:30.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:28:30.240 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:28:30.243 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:28:30.243 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-02-20T19:28:59.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8449669093427417) > weight (0.25) 2021-02-20T19:28:59.519 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:28:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:28:59.521 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:28:59.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T19:28:59.511+0000 7f6c4f6d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91403) UID: 1000 2021-02-20T19:28:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:29:07.521 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:29:07.522 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:29:07.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:29:07.523 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:29:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:29:07.524 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:29:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:29:07.528 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-02-20T19:29:39.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7593450833880381) > weight (0.25) 2021-02-20T19:29:39.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2651739963483042) > weight (0.25) 2021-02-20T19:29:39.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8589860044868947) > weight (0.25) 2021-02-20T19:29:39.250 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-02-20T19:29:39.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-02-20T19:29:39.251 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:29:39.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:29:39.246+0000 7f138a575700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 90365) UID: 1000 2021-02-20T19:29:39.653 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:29:47.251 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-02-20T19:29:47.252 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:29:47.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:29:47.253 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-02-20T19:29:47.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-02-20T19:29:47.253 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-02-20T19:29:47.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-02-20T19:29:47.256 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-02-20T19:30:09.529 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7804333373415759) > weight (0.25) 2021-02-20T19:30:09.529 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:30:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:30:09.530 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:30:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T19:30:09.526+0000 7f2e72bcf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91889) UID: 1000 2021-02-20T19:30:09.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:30:13.531 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:30:13.531 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:30:13.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:30:13.532 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:30:13.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:30:13.533 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:30:13.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:30:13.536 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-02-20T19:31:03.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9969885033867291) > weight (0.25) 2021-02-20T19:31:03.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4441417188591855) > weight (0.25) 2021-02-20T19:31:03.258 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-02-20T19:31:03.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-02-20T19:31:03.259 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:31:03.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:2021-02-20T19:31:03.253+0000 7f26b41ba700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 91213) UID: 1000 2021-02-20T19:31:03.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:31:05.260 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-02-20T19:31:05.260 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:31:05.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:31:05.261 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-02-20T19:31:05.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-02-20T19:31:05.262 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-02-20T19:31:05.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-02-20T19:31:05.265 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-02-20T19:31:27.536 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:31:27.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:31:27.552 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-02-20T19:31:27.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:31:27.548+0000 7f6597d12700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 91697) UID: 1000 2021-02-20T19:31:27.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:31:28.553 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:31:28.554 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:31:28.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:31:28.554 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:31:28.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:31:28.555 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:31:28.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:31:28.558 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-02-20T19:32:28.268 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8532362640455555) > weight (0.25) 2021-02-20T19:32:28.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9777094176831554) > weight (0.25) 2021-02-20T19:32:28.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5144920966508447) > weight (0.25) 2021-02-20T19:32:28.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4849418704420574) > weight (0.25) 2021-02-20T19:32:28.271 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-02-20T19:33:14.559 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7505727660596176) > weight (0.25) 2021-02-20T19:33:14.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9931798956064931) > weight (0.25) 2021-02-20T19:33:14.566 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-02-20T19:33:14.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-02-20T19:33:14.567 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:33:14.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:2021-02-20T19:33:14.562+0000 7fb630f0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 91597) UID: 1000 2021-02-20T19:33:14.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:33:23.568 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-02-20T19:33:23.569 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:33:23.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:33:23.570 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-02-20T19:33:23.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-02-20T19:33:23.570 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-02-20T19:33:23.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-02-20T19:33:23.573 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-02-20T19:34:16.272 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47818298945877846) > weight (0.25) 2021-02-20T19:34:16.273 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:34:16.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:34:16.274 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:34:16.275 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:34:16.272+0000 7f07490fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 91312) UID: 1000 2021-02-20T19:34:16.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:34:17.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.520+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270aac80 [3/16fde910-cc05-4991-83f7-5dc8ef8964ca] start: already running 2021-02-20T19:34:17.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.520+0000 7faf3d2fa700 -1 rbd::mirror::ImageReplayer: 0x5588270aaf00 [3/1cf3588b-5776-4e0d-8431-9fd0ea755403] start: already running 2021-02-20T19:34:17.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.524+0000 7faf3d2fa700 -1 rbd::mirror::ImageReplayer: 0x5588270ab180 [3/3f753aad-e91c-4581-b56d-0bf29964e7db] start: already running 2021-02-20T19:34:17.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270ab400 [3/5d895514-421a-41d1-8f87-96dea12501c8] start: already running 2021-02-20T19:34:17.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270ab900 [3/7cc83948-bf26-4ed7-a142-2db0f2786e6e] start: already running 2021-02-20T19:34:17.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270ab680 [3/80a8b728-6231-4700-be28-ff3940747d5f] start: already running 2021-02-20T19:34:17.534 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270abe00 [3/828e1151-fa83-405f-a6cf-ec7c06ae0dee] start: already running 2021-02-20T19:34:17.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3caf9700 -1 rbd::mirror::ImageReplayer: 0x5588270ac080 [3/a63c5ffe-a120-4d43-81ef-8e76b8e61d58] start: already running 2021-02-20T19:34:17.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi114.stderr:2021-02-20T19:34:17.528+0000 7faf3d2fa700 -1 rbd::mirror::ImageReplayer: 0x5588270aca80 [3/cad7127b-81aa-4ec0-a7a8-b2eafde0ec5d] start: already running 2021-02-20T19:34:21.275 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:34:21.276 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:34:21.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:34:21.277 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:34:21.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:34:21.277 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:34:21.280 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:34:21.281 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-02-20T19:35:06.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7553326568157487) > weight (0.25) 2021-02-20T19:35:06.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31600406090298105) > weight (0.25) 2021-02-20T19:35:06.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.94813024042693) > weight (0.25) 2021-02-20T19:35:06.594 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T19:35:06.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T19:35:06.601 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-02-20T19:35:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T19:35:06.595+0000 7fa716d90700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 85484) UID: 1000 2021-02-20T19:35:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:35:15.602 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T19:35:15.612 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:35:15.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:35:15.613 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T19:35:15.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T19:35:15.614 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T19:35:15.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T19:35:15.617 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-02-20T19:35:50.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5240722433868389) > weight (0.25) 2021-02-20T19:35:50.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5931118396483581) > weight (0.25) 2021-02-20T19:35:50.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9290686122830097) > weight (0.25) 2021-02-20T19:35:50.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2753771958833121) > weight (0.25) 2021-02-20T19:35:50.283 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-02-20T19:37:06.618 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6563662940918447) > weight (0.25) 2021-02-20T19:37:06.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3832068236801457) > weight (0.25) 2021-02-20T19:37:06.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.761241861608448) > weight (0.25) 2021-02-20T19:37:06.855 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-02-20T19:37:06.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-02-20T19:37:06.857 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:37:06.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:2021-02-20T19:37:06.853+0000 7f3fa721b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 92564) UID: 1000 2021-02-20T19:37:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:37:10.857 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-02-20T19:37:10.858 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:37:10.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:37:10.859 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-02-20T19:37:10.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-02-20T19:37:10.859 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-02-20T19:37:10.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-02-20T19:37:10.862 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:37:11.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8187878640237349) > weight (0.25) 2021-02-20T19:37:11.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8195944097786324) > weight (0.25) 2021-02-20T19:37:11.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6895434789909338) > weight (0.25) 2021-02-20T19:37:11.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.36674675108380717) > weight (0.25) 2021-02-20T19:37:11.285 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-02-20T19:38:55.863 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7308746665073311) > weight (0.25) 2021-02-20T19:38:55.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5447486833117189) > weight (0.25) 2021-02-20T19:38:55.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6815653389201195) > weight (0.25) 2021-02-20T19:38:55.865 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6684138789701197) > weight (0.25) 2021-02-20T19:38:55.865 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:38:56.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4760031630417376) > weight (0.25) 2021-02-20T19:38:56.286 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:38:56.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:38:56.287 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-02-20T19:38:56.288 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:38:56.286+0000 7f8f7d333700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 92470) UID: 1000 2021-02-20T19:38:56.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:39:01.288 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:39:01.288 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:39:01.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:39:01.289 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:39:01.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:39:01.290 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:39:01.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:39:01.293 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-02-20T19:40:30.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7680431575329001) > weight (0.25) 2021-02-20T19:40:30.872 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-02-20T19:40:30.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-02-20T19:40:30.873 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-02-20T19:40:30.874 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-02-20T19:40:30.874 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:40:30.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:2021-02-20T19:40:30.868+0000 7f00744d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 92081) UID: 1000 2021-02-20T19:40:31.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:40:36.294 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:40:36.295 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:40:36.295 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-02-20T19:40:36.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:40:36.292+0000 7f8c952c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 91796) UID: 1000 2021-02-20T19:40:36.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:40:36.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:40:36.977 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-02-20T19:40:36.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-02-20T19:40:36.977 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-02-20T19:40:36.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-02-20T19:40:36.980 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-02-20T19:40:44.296 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:40:44.302 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:40:44.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:40:44.303 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:40:44.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:40:44.305 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:40:44.311 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:40:44.311 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-02-20T19:41:42.981 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-02-20T19:41:43.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-02-20T19:41:43.007 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:41:43.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:2021-02-20T19:41:43.003+0000 7f4202b2b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 92272) UID: 1000 2021-02-20T19:41:43.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:41:47.007 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-02-20T19:41:47.008 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:41:47.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:41:47.009 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-02-20T19:41:47.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-02-20T19:41:47.009 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-02-20T19:41:47.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-02-20T19:41:47.012 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-02-20T19:41:47.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34122170689531317) > weight (0.25) 2021-02-20T19:41:47.313 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-02-20T19:41:47.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-02-20T19:41:47.313 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-02-20T19:41:47.314 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:2021-02-20T19:41:47.311+0000 7f5929db3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 92763) UID: 1000 2021-02-20T19:41:47.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:41:49.314 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-02-20T19:41:49.315 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:41:49.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:41:49.315 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-02-20T19:41:49.316 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-02-20T19:41:49.316 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-02-20T19:41:49.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-02-20T19:41:49.319 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-02-20T19:43:25.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8002784563146224) > weight (0.25) 2021-02-20T19:43:25.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9709546003841653) > weight (0.25) 2021-02-20T19:43:25.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6099151253891532) > weight (0.25) 2021-02-20T19:43:25.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.37242155550553124) > weight (0.25) 2021-02-20T19:43:25.015 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-02-20T19:43:43.320 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:43:43.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:43:43.322 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-02-20T19:43:43.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:43:43.320+0000 7f14d7ab0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 92947) UID: 1000 2021-02-20T19:43:43.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:43:50.323 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:43:50.324 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:43:50.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:43:50.324 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:43:50.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:43:50.325 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:43:50.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:43:50.330 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-02-20T19:45:12.331 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-02-20T19:45:12.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-02-20T19:45:12.343 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-02-20T19:45:12.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:2021-02-20T19:45:12.342+0000 7f290a9f9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 93237) UID: 1000 2021-02-20T19:45:12.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi114.stderr:daemon-helper: command failed with exit status 1 2021-02-20T19:45:16.344 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-02-20T19:45:16.344 INFO:teuthology.orchestra.run:waiting for 600 2021-02-20T19:45:16.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-02-20T19:45:16.345 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-02-20T19:45:16.345 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-02-20T19:45:16.346 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-02-20T19:45:16.348 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-02-20T19:45:16.349 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-02-20T19:45:20.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5926252483225096) > weight (0.25) 2021-02-20T19:45:20.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5399401920396393) > weight (0.25) 2021-02-20T19:45:20.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9529656269975686) > weight (0.25) 2021-02-20T19:45:20.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9111824432744169) > weight (0.25) 2021-02-20T19:45:20.017 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-02-20T19:45:47.392 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:45:47.389+0000 7f803c4bb700 -1 reset not still connected to 0x55acb8cb1dc0 2021-02-20T19:45:47.392 INFO:tasks.ceph.cluster2.osd.2.smithi114.stderr:2021-02-20T19:45:47.389+0000 7f803c4bb700 -1 reset not still connected to 0x55accd22d1e0 2021-02-20T19:45:51.460 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-02-20T19:45:51.464 DEBUG:teuthology.task.console_log:Killing console logger for smithi106 2021-02-20T19:45:51.465 DEBUG:teuthology.task.console_log:Killing console logger for smithi114 2021-02-20T19:45:51.466 DEBUG:teuthology.exit:Finished running handlers